DrToller.Framework
Declarative execution framework for data and ML workloads.DRTML describes the logical shape of a workload; the Framework validates, compiles, and executes it across interchangeable compute, storage, parallelism, and observability backends. Humans extend primitives; humans and AI compose them into pipelines.
pip install drtoller
drtoller version
drtoller init my-pipelineDRTML expresses the logical shape of a data or ML workload: datasets, feed, params, metrics, and execution mode. The Framework parses, validates, compiles, and executes that contract — it is not merely configuration bolted onto tasks.
Humans implement and register Framework primitives. Humans and external AI discover them in the capability catalog and compose DRTML. Structured validation supports generate→repair loops. The runtime does not embed an LLM.
Steps interact through explicit execution contracts. Storage, parallelism, checkpoints, metrics emit, UI, and hosts stay in the Framework — including stateful modes such as embedding training.
Parallelism and storage are contracts over registries. Ray is a current parallelism backend — not the Framework’s identity. Parquet, PostgreSQL, and Qdrant sit behind the same session adapters.
Prometheus series and Grafana views live in the same DRTML contract as the run. Framework emits progress, resources, and saturation — steps do not call prometheus_client.
Register algorithms, metrics, evaluation methods, storage and parallelism backends, execution modes, and integrations through the Extension SDK — the open Framework is meant to be extended, not forked as a black box.