Fragments and include

Why fragments

Keep the main manifest focused on identity and execution. Put defaults, UI, metrics, datasets and mode contracts into fragments.

Include syntax

Each include entry is a mapping. String lists are rejected.

include:
  - {path: fragments/params.drtml, merge: params_defaults}
  - {path: fragments/ui.drtml, merge: ui}
  - {path: fragments/metrics.drtml, merge: metrics}
  - {path: fragments/storage.drtml, merge: storage}
  - {path: fragments/datasets/input.drtml, merge: datasets}

path is resolved relative to the including file. Nested includes are allowed; cycles and missing files fail load.

Allowed merge targets

• ui, metrics, processing, orchestrator
• storage, datasets, mapping_read, registry, params_defaults
• mapping, reduce_merge, execution, postgres
• embed_train, embed_infer

Merge semantics:

• mapping-like sections deep-merge;
• datasets deep-merge by dataset id;
• registry concatenates lists;
• mapping / reduce_merge also carry pattern when present;
• metrics also merges sibling grafana.

Typical fragment set

Fragment	Merge target	Contents
params.drtml	params_defaults	defaults
ui.drtml	ui	Streamlit form
metrics.drtml	metrics	Prometheus + views (+ grafana)
storage.drtml	storage	bindings / resume
datasets/*.drtml	datasets	one or more datasets
processing.drtml	processing	processor wiring
orchestrator.drtml	orchestrator	DAG / connections / XCom

UI ↔ params

Every form field must change a live merged param and have a runtime/domain reader. Dead controls are forbidden.

Metrics views

Grafana panels are declared in metrics.views. CPU and RAM panels are mandatory. JSON codegen is server-only.