Skip to content

PipelineStep: add try/catch and expose failing folder path on error #16

@paACode

Description

@paACode

PipelineStep — wrap run() with try/catch and report failing folder path

Problem

Currently PipelineStep.run() has no error handling. When a step fails mid-way through iterating over folders, the pipeline crashes immediately with no information about which folder caused the failure.

Proposed Solution

Split run() into two methods:

  • Rename the existing run() abstract method to _run() — subclasses implement this
  • Add a new run() as a wrapper that handles errors and logging
from typing import final

class PipelineStep(ABC):

    def run(self) -> bool:
        """Public entry point. Wraps _run() with error handling."""
        print(f"Starting Step: {self.name} ⏳")
        try:
            self._run()
            print(f"Finished Step: {self.name} ✅")
            return True
        except Exception as e:
            print(f"Failed Step: {self.name} ❌ — {e}")
            raise

    @abstractmethod
    def _run(self) -> None:
        """Actual step logic. Subclasses implement this."""

Inside each subclass _run(), the per-folder loop should catch and re-raise with the folder path:

def _run(self) -> None:
    for input_folder, output_folder in zip(self.input_folders, self.output_folders):
        try:
            # ... step logic ...
        except Exception as e:
            raise RuntimeError(f"Failed at folder: {input_folder}") from e

Expected Output

Failed Step: BuildDigitalSurfaceModel ❌ — Failed at folder: D:/data/mjolnir/re112o_250610/02_processing

Tasks

  • Rename run()_run() in PipelineStep base class
  • Add run() wrapper with try/catch to base class
  • Update all subclasses: rename run()_run() and add per-folder try/catch

Out of Scope

How the pipeline orchestrator handles step failures (skip vs. abort, retry, reporting) is not part of this issue.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions