Dispatch

Dispatch is the front door of a run. Something outside the step (Streamlit form, Airflow task, local CLI) builds a RunContext, points at a DRTML manifest, and asks the framework to execute it. The step does not invent its own “main”.

from drtoller.framework.processing.dispatch import run_from_manifest

outcome = run_from_manifest(manifest_path, ctx, user_params=…)

Thin step hooks (run_partition_loop, run_mapping, …) are convenience wrappers that end in the same runners.

What happens

flowchart LR
  A["load_plugins"] --> B["Load DRTML + merge params"]
  B --> C["emit_run_start"]
  C --> D["Optional metrics setup"]
  D --> E{"Known pattern?"}
  E -->|yes| F["Pattern runner"]
  E -->|no| G["execution.entry"]
  F --> H["Finalize metrics"]
  G --> H
  H --> I["emit_run_end"]

1. load_plugins() — Extension SDK entry points (drtoller.plugins) register backends / runners / methods before dispatch.
2. Load DRTML v4, lower to an internal manifest, merge params_defaults with UI/Airflow overlays.
3. emit_run_start (Extension SDK hook).
4. If metrics are enabled, compile the storage plan and set up Prometheus handles.
5. If the lowered pattern is registered (mapping, reduce_merge, dataset_metrics, evaluation), call that runner.
6. Otherwise import execution.entry (partition, embed train, custom, …).
7. Always finalize metrics in finally, then emit_run_end.

Full lifecycle diagrams: How a run executes. Which run.type maps where: Modes.

Registries (extend without if-forests)

Built-in pattern and mode runners live in runtime/pattern_runners.py (ensure_builtin_runners). processing/dispatch orchestrates the run and re-exports the register/get APIs for compatibility. Prefer the public SDK:

from drtoller.framework.ext import register_pattern_runner, register_mode_runner

register_pattern_runner("my_pattern", my_run)
register_mode_runner("my_mode", my_run)

Kind	Examples	Meaning
pattern	mapping, reduce_merge, dataset_metrics, evaluation	Dedicated DRTML block + shared runner
mode	partition_loop, …	Legacy / lowered mode keys for the same idea

Lookup uses get_pattern_runner / get_mode_runner — do not treat private dict names as API.

Do not add new branches inside run_from_manifest. Vertical recipe: New execution mode. Registries map: Extension registries.