mapping mode

Mapping answers a simple problem: the same real-world entity appears under different run-local ids, and later steps need one stable identity plus a table that maps “this run’s id → canonical id”.

The step does not implement the merge algorithm. It declares inputs/outputs in DRTML; framework loads upstream artifacts, merges according to the mapping plan, writes registry + crosswalk, and emits metrics. Downstream steps typically prefetch an id index (mapping_read) and only remap columns. Nested probe evaluate can set mapping.assignment_upstream_runs so the child reads spilled assignment parquet instead of a pickled map.

What you declare

pattern: mapping
execution:
  entry: drtoller.framework.processing.mapping_mode.run:execute_mapping_build
mapping:
  entry: drtoller.framework.processing.mapping_mode.run:execute_mapping_build
  inputs: {entities: source_entities}
  outputs: {registry: canonical_registry, crosswalk: run_crosswalk}
  registry_id_column: canonical_id
  crosswalk_id_column: canonical_id

Dispatch routes this to run_mapping. Details of identity keys, hash, and counts live in the mapping DRTML / identity blocks on datasets.

What framework does

1. Compile a MappingPlan from the manifest.
2. Open a StorageSession for the run.
3. Run framework.mapping.build (preload → merge → materialize).
4. Emit post-build metrics.
5. Return finalized storage manifests.

Useful pieces when you dig deeper: mapping/build/ (orchestration), mapping/remap.py / mapping/read.py (id helpers and prefetch), contracts/mapping_build.py (result DTOs).

What the step owns

DRTML (and optional thin hooks that re-export the framework entry). No homegrown merge in Python. Cookbook: New mapping step.