July 2026
Product notes for July 2026. Later months: August 2026, September 2026. Index: What’s new.
Week of July 25
Section titled “Week of July 25”- Probe is a product loop. The runtime tries candidates; the step chooses the winner. Nested evaluate shares the parent’s workers instead of multiplying them. Final publish uses the full stream; trials can run on a subset.
- Cluster at corpus scale. Fit and assign stream vector batches. Peak RAM is the model plus one batch, not the full matrix. The same path serves many probe
Kvalues once sticky cache is on (see August). - Evaluation covers clusters and graphs, not only vector geometry: quality of assignments, topology of edges.
- Dashboards show probe phase while the first prepare is still running, so Grafana is not empty for the long opening stretch.
Details: Probe loop, Streaming cluster, evaluation, Grafana.
Week of July 23
Section titled “Week of July 23”- Embed train on many workers is real. Weights shard across workers; each batch updates the shards that own those rows. A single shared table is no longer pretending to be distributed.
- Large keyed joins shuffle, then reduce per bucket. Producers write stable buckets; reduce sums each bucket in parallel. That is the scale-out path, not “one task per dataset”.
- Reads stream. Production scans do not load a whole run into RAM. Optional grouped sums accumulate on the stream.
- Occurrence pooling is batched, so embed infer does not walk the vocab one row at a time.
Details: embed_train, embed_infer, reduce_merge, StorageSession.
Week of July 21
Section titled “Week of July 21”- Corpus-growth curves you can plot. Offline metrics jobs distinguish a final exact point, hashed buckets, and document-progress milestones. Grafana XY uses corpus fraction vs the metric.
- The same observation table can take embed and evaluation points, so dashboards stay on one shape.
- Postgres migrate is boring on purpose. Managed objects follow a fixed naming scheme; leftovers from old schemas are dropped. Dashboards keep calling the same procedures.
Details: dataset_metrics, Grafana, PostgreSQL.