Install

Framework package

Status: Developer preview 0.1.4 (Apache-2.0). APIs may change between 0.x minors — see What’s new and the Framework CHANGELOG.md.

pip install drtoller
# optional extras:
#   pip install "drtoller[ray,pg,s3,ui,api]"
drtoller version

PyPI package and CLI: drtoller (pip install drtoller, then drtoller …). Import: drtoller.framework. Extra api is the run control-plane HTTP server (drtoller serve api).

From a source checkout (Framework repo root):

pip install -e ".[dev]"
# optional extras: ray, pg, s3, ui, api, docs

Core dependencies include PyArrow, Pydantic, PyYAML and Prometheus client support. Install optional Ray, PostgreSQL, S3/MinIO, Streamlit, and run-API extras only where those integrations are used.

CLI

drtoller version
drtoller validate path/to/step.drtml
drtoller validate path/to/step.drtml --compile
drtoller validate path/to/step.drtml --format json
drtoller plan path/to/step.drtml --format json
drtoller init my-pipeline
drtoller inspect --format json
drtoller inspect --level 1
drtoller ray status
drtoller deploy env --list-providers
drtoller deploy env --provider minio
drtoller run path/to/step.drtml --run-id RUN_ID
drtoller run path/to/step.drtml --config run.yaml --param KEY=VALUE
drtoller run path/to/step.drtml --via airflow --format json
drtoller serve api --port 8088
drtoller ls path/to/step.drtml
drtoller ls path/to/step.drtml --format json
drtoller clear path/to/step.drtml --run-id RUN_ID
drtoller clear path/to/step.drtml --run-id ID1,ID2 --yes
drtoller clear path/to/step.drtml --all
drtoller clear path/to/step.drtml --all --yes

ls lists run_ids with parquet size/object count and PG row counts (one scan). clear plans deletes (parquet prefix / DELETE … WHERE run_id). Pass --run-id (comma-separated) or --all (every id from ls), not both. Default is dry-run; --yes executes. Never truncates the table or touches schema_version. Optional --datasets a,b, --include-inputs, --include-metrics (sibling *_metrics step plus live metric tables), --metrics-manifest PATH. --format json for scripts.

run uses the same submit path as HTTP and Streamlit. --config is YAML/JSON (run_id, artifacts_root, params); flags override the file. --via local (default) runs on this host; --via airflow triggers the step DAG. drtoller serve api is the HTTP control plane (POST/GET /v1/runs). See Run API.

If host Python cannot import drtoller.framework and pyarrow, the Platform wrapper runs these in airflow-worker (DRTOLLER_FW_BACKEND=auto|host|docker).

Capability discovery and repairable validation: AI & Automation. AI bootstrap: AI quickstart. Ray multi-host / object store: Ray. Stack ops (up, k8s up, k8s down, install, ray bake): CLI operations.

Smoke tests (no step packs)

python -m pytest -q examples/hello-step examples/hello-plugin examples/hello-mapping

Framework-only unit tests exclude step-pack, platform, and integration markers. The exact pytest invocation is in the Framework repo CONTRIBUTING.md. Contract tests that load an OSS step pack such as drtoller.edge are opt-in.

Releases / PyPI

Cutting a version: tag vX.Y.Z on DrToller.Framework (see package docs/PUBLISHING.md). GitHub Release + Trusted Publishing upload the wheel to PyPI.

Step package

A step is a separate installable Python package containing importable processor code and packaged *.drtml files:

pip install acme-pipeline-steps
export DRTOLLER_STEPS_ROOTS=/opt/acme-pipeline-steps/manifests

Official OSS steps live in DrToller.Edge (an example open step pack). Additional step packages use the same DRTOLLER_STEPS_ROOTS / DRTOLLER_STEPS_ROOTS_FILE contract. Discovery does not scan sibling checkout paths automatically.

Deployment environment

A Framework workload needs an environment that provides the runtime (storage, compute, observability, and related capabilities). DrToller.Platform is that layer. Product page: platform.drtoller.com.

The current default is a self-hosted Compose environment plus operator CLI (drtoller up, drtoller update, …). That composition is a shipped reference, not a Framework requirement: you can change, replace, or omit services if the capabilities the workload needs are provided another way.

Typical sibling checkout:

/opt/DrToller.Framework
/opt/DrToller.Edge
/opt/DrToller.Platform

Step code stays independent of which services are up; DRTML selects backends and integrations.

See the Deploy section:

Dashboard generation and PostgreSQL DDL sync are deployment-only operations — not part of step runtime.

This documentation site

Canonical docs are this Starlight site in DrToller.com (framework/docs/):

cd framework/docs
npm install
npm run dev
npm run build

Deployed at https://framework.drtoller.com/docs. Company landing: drtoller.com.