Integrations map

Hosts differ; the step does not. Streamlit, Airflow, the local CLI, and the run HTTP API each collect params, build a RunContext, and call dispatch. Ray connection is a transport concern for workers — not a second execution API for domain code.

User / Scheduler / HTTP
  → integration (params overlay + RunContext)
  → processing.dispatch
  → pattern runner or execution.entry

Steps do not import integration/* (except thin hooks that call framework APIs). Worker submit/pool code lives under processing/.../parallel/backends/, not in integration.

Package map

Subpackage	Role
`ui/`	Streamlit launcher + DRTML-driven forms
`api/`	Run control-plane: `submit_run`, FastAPI `POST/GET /v1/runs`
`run/`	Shared `prepare_run` (param merge + `RunContext`)
`airflow/`	DAG discover, `run_task`, orchestrator params
`local/`	CLI runner without Airflow
`ray/`	`ensure_ray_connected`, `preflight_ray`, `wait_ray_worker_cpus`, `ensure_raycluster_shape` (`cluster_shape` + kubectl in `cluster_tune`); client address from head Service LB when kubectl is available
`env.py`	env helpers / DSN mapping

Details: Streamlit, Airflow, Run API, Ray.