Add a dataset metric method

A dataset metric method computes one value from dataset state. Its DRTML alias and Prometheus metric name are separate concerns.

1. Specify semantics

Define:

• required entity/mass/related columns;
• supported cadence and accuracy;
• whether novelty samples, growth state, threshold or sampling knobs are required;
• supported backends;
• behavior for empty input and missing values.

2. Implement a pure formula when applicable

def mean_positive_mass(rows, column, **_):
    values = [float(row[column]) for row in rows if row.get(column) is not None and float(row[column]) > 0]
    return None if not values else sum(values) / len(values)

Place it in algorithms/dataset_metric_formulas.py or a focused algorithm module. It must not open a session or emit Prometheus.

3. Register capability metadata

register_method(MetricMethod(
    name="mean_positive_mass",
    python=mean_positive_mass,
    postgres_proc_metric="mean_positive_mass",
    qdrant_metric=None,
    needs_novelty=False,
    accepts_sampling_knobs=False,
    requires_threshold=False,
    needs_growth_state=False,
    requires_related=False,
    supported_modes=("checkpoint_exact", "final_exact"),
))

Avoid a new if method == branch. Registry metadata drives validation and backend selection.

4. Add backend execution

For PostgreSQL, extend the generic metric snapshot procedure’s whitelisted metric implementation and deployment SQL tests. Do not create a procedure per step/dataset and do not execute DDL at run time.

For Qdrant, filling qdrant_metric alone is not enough: add or extend a runtime consumer (today geometry is a dedicated embed-train/Qdrant path). A method may intentionally support only one backend path.

5. Declare it in DRTML

datasets:
  counts:
    metric_defs:
      positive_mean:
        method: mean_positive_mass
        column: count
        unit_column: document_id
        cadence: final
        accuracy: exact

metrics:
  prometheus:
    - name: step_positive_mass_mean
      kind: gauge
      labels: [run_id, step_id, stage, dataset]
      source: dataset_metric
      dataset: counts
      metric: positive_mean

metric: positive_mean references the alias under metric_defs, not the method name.

6. Test all layers

• Formula: normal, duplicate, null and empty rows.
• Registry: resolution, capabilities and duplicate registration behavior.
• DRTML: supported/unsupported cadence, accuracy, threshold/related requirements.
• Backend: Python/PG/Qdrant result parity where multiple paths exist.
• Telemetry: alias resolves to the declared Prometheus name.
• Dashboard: generated panel query uses the emitted metric.

Do not manually emit the new method from processor code.