DrToller

Positioning

What problem does Framework solve?

DrToller.Framework is a declarative execution framework for data and ML workloads.DRTML expresses the logical shape of a workload as amachine-readable execution contract, which the Framework validates, compiles, and executes across interchangeable compute, storage, parallelism, and observability backends.

A machine-readable execution contract for data workloads.

Developer preview · August 2026. This page explains abstraction layers, not product rankings. These technologies often coexist in one stack.

The problem

Data and ML systems usually mix what the workload means with how it runs: storage paths, worker transport, metrics emit, and host scheduling end up inside domain code or informal task scripts. That makes workloads hard to validate, hard to reuse across backends, and hard for tools (including AI) to compose safely.

DrToller separates the logical workload (DRTML + domain logic) fromphysical infrastructure (Framework backends and Platform hosts).

The abstraction

Traditional stacking

Application code
      ↓
Task
      ↓
Orchestrator
      ↓
Compute engine
      ↓
Storage

DrToller

Domain logic
      ↓
DRTML execution contract
      ↓
Compiled execution plan
      ↓
DrToller.Framework
  ├── compute / modes
  ├── storage backends
  ├── parallelism backends
  ├── metrics / telemetry
  └── host integrations

Purpose: keep the logical workload independent of the physical infrastructure used to execute it.

Two-sided programming model

Humans extend the Framework. AI and humans compose the Framework’s primitives into pipelines.

Human developers
      │ implement
      ▼
Framework primitives  (algorithms, modes, backends, metrics, …)
      │ registered + described
      ▼
Machine-readable capability catalog
      │
      ├───────────────┐
      ▼               ▼
    Human             AI (external)
      │               │
      └───────┬───────┘
              ▼
            DRTML
              ▼
          validation
              ▼
        compiled plan
              ▼
           execution

Distill: humans expand what the Framework can do; AI expandswhat users can build with those capabilities — by generating and repairing DRTML against structured contracts. The runtime does not call an LLM.

Available today

DRTML, typed plans, registries, capability catalog, composition checks, structured validation (validate --format json), and a no-LLM generate→repair example in the Framework repo.

External / emerging

Production LLM agents that draft DRTML from natural language liveoutside this package. The Framework supplies the machine-readable surface they need.

How the abstraction differs

Not “DrToller versus Spark.” Different questions — often answered together.

Airflow · Prefect · Dagster

When should this workload run, and what depends on what?

Scheduling, dependencies, orchestration

Spark

How do I compute SQL / DataFrames at warehouse scale?

Distributed SQL and dataframe computation

Ray

How do I distribute Python compute and actors?

Distributed execution / compute transport

ClickHouse (and similar)

How do I store and query analytical tables?

Analytical database

LangChain-class systems

How do models, tools, and agents form an application?

LLM / application / agent orchestration

DrToller.Framework

What is this data/ML workload as a typed contract, and how should that contract execute?

Declarative workload contract, execution semantics, infrastructure separation, composable primitives

Layer detail

Orchestration

Airflow · Prefect · Dagster

Airflow answers: when should this workload run and what depends on what?
DrToller answers: what is this workload and how should its logical execution contract be realized?

An Airflow DAG can invoke a DrToller workload. InDrToller.Platform, Airflow is a supported host — complement, not a competitor to replace.

Distributed SQL / DataFrames

Spark

Spark is a strong choice for distributed SQL, DataFrame, and warehouse-scale ETL workloads. DrToller addresses a different abstraction: expressing data-processing workloads as declarative, typed execution contracts while keeping domain logic independent from physical execution infrastructure.

We do not claim to replace Spark or to be faster than Spark. Architecture stays open to Spark-class engines as future or adjacent compute where that is the right fit.

Compute transport

Ray

Ray is currently one of DrToller’s execution/parallelism backends — not “DrToller is a Ray framework.”

DrToller
   ↓
parallel / execution contract
   ↓
backend registry
   ↓
Ray (today)

Contributors can register alternative parallelism backends through the Framework’s registries. We do not promise specific future backends here.

LLM / agents

LangChain-class systems

LLM/agent frameworks orchestrate model interaction, tools, and application workflows. DrToller composes data workloads: declarative execution, distributed processing, storage abstraction, and machine-readable contracts.

AI agent (external)
    ↓
generates DRTML
    ↓
DrToller validates and executes

DrToller does not need to contain an LLM.

What to evaluate

Execution contract

DRTML is a declarative workload language: parsed, validated, semantically checked, compiled, inspected, and executed. It is not “YAML config bolted onto tasks.”

Infrastructure separation

Domain logic interacts with the Framework through explicit contracts. Storage, parallelism, metrics, UI, and hosts stay outside step packages.

Human-extensible, machine-composable

Humans register new primitives (algorithms, backends, modes, metrics). Humans and external AI compose those primitives into DRTML. The Framework does not embed an LLM.

Interchangeable backends

Parallelism, storage, and related concerns are contracts over registries. Ray is a current parallelism backend — not the Framework’s identity.

Separation of concerns

Domain logic
     ↓
DRTML / execution contract
     ↓
Framework
  ├── execution modes
  ├── parallelism backends
  ├── storage backends
  ├── metrics / telemetry
  ├── host integrations
  └── deployment (Platform)

Domain developers should not re-implement infrastructure inside every step. Steps interact through explicit contracts; the Framework owns feed, write, checkpoint, metrics emit, and worker transport. Stateful modes (for example embedding training) still keep infrastructure concerns in the Framework — domain code does not open storage or Prometheus.

Extensibility (first-class)

The Framework is intentionally open source and designed to be extended. Developers can add algorithms, metric methods, evaluation methods, storage backends, database integrations, parallelism backends, execution primitives, and host integrations through registries and the plugin system — not by forking a closed engine.

Next

Product overviewDRTML overviewInstall