dataset_metrics

pattern: dataset_metrics evaluates declared count artifacts outside the producing step. It is distinct from live/checkpoint datasets.*.metric_defs: this pattern writes metric-point datasets over one or more upstream runs.

Minimal pattern

drtml: 4
step: token_metrics
schema: token_metrics_v1

run:
  type: dataset_metrics
  executor: python                 # python | postgres
  upstream_runs_param: metrics.upstream_runs
  bucket_column: corpus_bucket
  bucket_count_param: compute.corpus_growth.bucket_count
  milestone_fractions_param: compute.corpus_growth.milestone_fractions
  order_seed_param: compute.corpus_growth.order_seed
  mode_param: metrics.mode
  milestone_step_param: metrics.milestone_step
  max_milestones_param: compute.corpus_growth.max_milestones
  jobs:
    - input: token_counts_by_bucket
      output: corpus_growth_metric_points
      mode: buckets                # buckets | final | milestones
      key_columns: [token_id]
      mass_column: count
      metrics:
        cardinality: {method: count_distinct, column: token_id}
        mass: {method: total_mass, column: count}
        heaps_beta: {method: vocabulary_growth_beta, column: count}

Public DRTML v4 uses run.type: dataset_metrics. The loader lowers run into the internal pattern/entry model; step manifests must not declare that internal representation directly.

Each job requires declared input/output datasets, at least one key column, and at least one registered dataset metric method. Endpoint, group/entity, and center/context columns are optional paired contracts for graph, cross-group, and reciprocity methods.

At runtime, metrics.mode selects jobs with the matching mode; a manifest may declare separate final and milestone jobs.

Job modes

Mode	Result	Ordering / evidence
buckets	cumulative corpus-growth points	stable-hash buckets
final	one exact merged point	all selected upstream runs
milestones	document-progress curve	document rows or PostgreSQL document-count evidence

Milestones from per-document counts

jobs:
  - input: lemma_document_count
    output: lemma_metric_points
    dataset_label: lemma_vocab
    mode: milestones
    evidence: document_counts
    document_column: doc_id
    key_columns: [lemma_token_id]
    mass_column: token_count
    metrics:
      cardinality: {method: count_distinct, column: token_count}
      new_in_sample: {method: assignment_window_new_count, column: token_count}

evidence: document_counts requires executor: postgres, a PostgreSQL input, mode: milestones, and document_column. The source must contain one count row per document/key. If an upstream run predates evidence support, rerun its producer.

Documents are stable-hash ordered using order_seed_param. metrics.milestone_step controls documents per point; max_milestones_param bounds the number of windows.

Executors

• python streams logical rows through the in-process accumulation engine and writes each job.output.
• postgres invokes generic deployed procedures and requires postgres_output referencing a PostgreSQL output dataset. Staged jobs clean run-scoped evidence in finally.
• PostgreSQL final jobs use a direct exact query when all selected methods advertise postgres_direct_final and the input is PostgreSQL; otherwise they use staging.

The runtime opens storage through StorageSession; neither executor performs DDL. PostgreSQL writes are bulk operations.

Metric points

All paths produce the same point rows: dataset label, metric name/value, corpus bucket/fraction, source runs, ordering policy/seed, and observation time. For milestone jobs, the bucket is the document count at the window end and corpus_fraction is progress through the ordered documents.

Output and dispatch

run_dataset_metrics validates the v3 pattern and delegates to dataset_metrics.entry. The builtin entry returns input-row, metric-point, per-dataset, manifest, and elapsed-time metadata.

Use Dataset metrics for the method registry and Add a metric method for extension rules.

Complete official example: Step01 Metrics.