Add an algorithm method
Use this recipe for embedding, clustering, and other pure kernels under framework/algorithms/.
For dataset metric formulas (metric_defs.method), use New metric method instead.
Boundary
Section titled “Boundary”| Framework owns | Domain / step owns |
|---|---|
| Math kernel, DTO result, method registry | When to call, probe loops, K choice, thresholds |
compute.* → ComputePlan kwargs |
Mapping results → logical ProcessResult rows |
| No I/O, no Prometheus, no DRTML parse | No sklearn/torch reimplementation of a registered method |
DRTML configures methods; it does not auto-execute arbitrary algorithms by name.
1. Choose the package
Section titled “1. Choose the package”| Family | Package | Registry |
|---|---|---|
| Embedding | algorithms/embed/ |
register_embed_method |
| Clustering | algorithms/cluster/ |
register_cluster_method |
| Evaluation formulas | algorithms/evaluation/ |
register_evaluation_method |
| Graph / profiles / information / stability | matching package | public package API (no root facade yet) |
Do not put kernels in processing/ or in a step package.
2. Implement data-in / result-out
Section titled “2. Implement data-in / result-out”Example: a second cluster method.
from typing import Any, Mappingimport numpy as npfrom drtoller.framework.algorithms.types import ClusterResult
def run_my_kmeans_v1( vectors: np.ndarray, row_ids: list[str], *, k: int, seed: int = 0,) -> ClusterResult: # pure computation only ... return ClusterResult( method_id="my_kmeans_v1", k=k, row_ids=list(row_ids), labels=labels, inertia=inertia, )
def my_kmeans_v1_from_params( vectors: np.ndarray, row_ids: list[str], params: Mapping[str, Any],) -> ClusterResult: return run_my_kmeans_v1( vectors, row_ids, k=int(params["k"]), seed=int(params.get("seed", 0)), )Rules:
- Accept arrays / DTOs; return a typed result (
ClusterResult,EmbedResult, …). - No
StorageSession, no files, no Prometheus. - No domain column names (
doc_id, lemma keys, …) inside the kernel. - Numeric knobs that steps tune must eventually map from
compute.*(see §4); do not invent silent fallbacks that diverge from DRTML defaults.
3. Register
Section titled “3. Register”# algorithms/cluster/__init__.py (or package import side-effect)from drtoller.framework.algorithms.registry import register_cluster_methodfrom drtoller.framework.algorithms.cluster.my_kmeans_v1 import my_kmeans_v1_from_params
register_cluster_method("my_kmeans_v1", my_kmeans_v1_from_params)Embedding:
from drtoller.framework.algorithms.registry import register_embed_methodregister_embed_method("my_embed_v1", my_embed_v1_from_params)Call sites stay registry-based:
from drtoller.framework.algorithms import cluster_with_k
result = cluster_with_k( vectors, row_ids, chosen_k, method_id="my_kmeans_v1", params={"seed": 42},)4. Wire compute.* (when the method is selected from DRTML)
Section titled “4. Wire compute.* (when the method is selected from DRTML)”- Add keys to step
params_defaults/ UI (namespacecompute.*). - Extend
ComputePlan/*_algorithm_kwargsindrtml/models/compute/if new knobs are needed. - Domain processor (or mode) reads the plan and passes
method_id+ kwargs into the registry API.
params_defaults: compute.cluster_method: my_kmeans_v1 compute.cluster_seed: 42Do not add framework policy params such as hidden compute.cluster_k when K selection is domain responsibility.
Caveat: production embed_train currently hard-requires sgns_v1 through SgnsVocabTrainer. Registering another embed method is enough for embed_rows, not automatically for embed_train.
5. Package APIs without a method registry
Section titled “5. Package APIs without a method registry”Graph / profile / information / stability kernels are imported from their packages:
from drtoller.framework.algorithms.graph import pmi_familyfrom drtoller.framework.algorithms.information import delta_hAdding one of these:
- Implement the pure function in the package.
- Export it from the package
__init__/ public module. - Document inputs/outputs in catalog.
- Do not invent a new root
algorithms.*facade method unless the family already uses one.
6. Tests
Section titled “6. Tests”Minimum:
- unit: empty input, invalid
k/ dims, deterministic seed; - registry: unknown
method_idraises with known names; - kwargs:
*_from_paramsmatches DRTML /ComputePlanmapping; - optional integration: processor or mode calls the method and writes logical rows via
ProcessResult(framework write path).
7. Docs checklist
Section titled “7. Docs checklist”- catalog — method id, inputs, outputs, params;
- invocation — if
compute.*surface changes; framework/FILES.mdwhen a new module lands.