Step vs framework boundaries

Step canon

Receive data → transform → return data. Transform logic only.

Framework owns: feed, StorageSession.write, flush, manifests, registry checkpoint, Prometheus, UI, Airflow, Ray transport.

Forbidden in the step (domain)

Any I/O: open, Path for data, shutil, ctx.storage.read_* / write
pyarrow, boto3/minio, psycopg, qdrant_client, HTTP to MinIO/DB
prometheus_client, RunMetrics, scrape endpoint
Airflow API, import ray, multiprocessing routing
Parsing DRTML / YAML / JSON manifests
Homegrown embed/cluster math (sklearn/torch/manual SGNS) — use framework.algorithms
Custom Streamlit widgets in hooks (forms come from ui.drtml)

Domain libraries required by the transform (for example spaCy in an NLP step) are allowed; their file/model lifecycle must still respect the processor contract.

Allowed in the step

Pure functions and in-memory registry plugins
ProcessResult / logical row-dicts
ProcessorBatch as the wire DTO at the batch_processor boundary
framework.mapping.remap / hash helpers
framework.algorithms (data in → result out; K policy is domain)
In hooks.py: resolve manifest_path via manifest_path_adjacent

Framework does not read `domain.*`

domain.* params are passthrough into the processor. Orchestration reads only runtime.*, storage.*, flush.*, compute.*, mapping.*, …

Dashboards

Every metrics.views.panels must include cpu_stat and ram_stat (step_cpu_percent, step_ram_used_mib).

CI guards

test_step_domain_boundaries.py — forbidden snippets in domain.
test_layer_import_boundaries.py — reverse import edges.