reduce_merge
Purpose
Section titled “Purpose”Fold partial outputs (often after multi-worker) into a final dataset per the DRTML reduce plan.
Layers
Section titled “Layers”| Layer | Role |
|---|---|
processing/reduce_merge/run.py |
lifecycle + metrics ticks |
storage/run/reduce_merge_* |
I/O / SQL merge |
| reduce build | algorithm |
Important (P0): storage must not import processing metrics.
Processing passes an on_input_ticks callback.
Pattern
Section titled “Pattern”pattern: reduce_mergeDispatch → run_reduce_merge.