Stack CLI

Install once per server:

cd /opt/drtoller.platform/stack
chmod +x bin/*.sh
sudo bin/install-stack-cli.sh

This installs only /usr/local/bin/drtoller (bin/drtoller.sh) and removes any legacy flat drt* symlinks. Scripts resolve the stack directory via readlink -f (works from any cwd). Non-default location: /etc/drtoller/stack.env with STACK_DIR=… and optional DOCKER_BIN=….

Framework subcommands (validate, plan, run, init, inspect, …) are forwarded to drtoller when the package is importable. See Framework docs/CLI_MIGRATION.md.

Command reference

Command	Script	When to use
`drtoller up`	`servstart.sh`	Full stack bring-up: pull, Postgres migrate, MinIO init, Airflow init, Ray build, Grafana sync
`drtoller up only SVC…`	partial `compose up -d`	Start specific services without pull/init
`drtoller down`	`servdown.sh`	`docker compose down` — stop and remove project containers
`drtoller down stop [SVC…]`	`compose stop`	Stop without removing containers
`drtoller restart [SVC…]`	`drtrestart.sh`	Restart Airflow + Ray + Streamlit (default), or named services
`drtoller update`	`drtupdate.sh`	After code deploy: PG migrate, Grafana sync, rebuild ray-head (+ optional `ray-worker` image), recreate Airflow + restart Streamlit/Prometheus
`drtoller update --git`	+ `git fetch/reset`	Same as deploy hook: sync repo then update
`drtoller update --minimal`	restart worker only	Fast path when only Python/DRTML on bind mount changed
`drtoller update --env`	delegates to `drtenvupdate.sh`	Env refresh + `compose up -d`
`drtoller env`	`ensure-env.sh`	Create `.env` from `env.example` if missing (usually called by `drtoller up`)
`drtoller env update`	`drtenvupdate.sh`	Replace `.env` from `env.example` (backup `.env.bak`), then `compose up -d`
`drtoller env update --merge`	merge mode	Append only new keys from `env.example`
`drtoller verify`	`verify-stack.sh`	Health: Redis, Postgres, MinIO, Airflow, compose Ray; if `RAY_ADDRESS=ray://…` — remote KubeRay + MinIO VPC
`drtoller fixperms`	`fix-artifact-perms.sh`	Fix group write on `/step01`…`/step05` and spaCy volume
`drtoller cloud bootstrap --provider digitalocean`	`cloud-do-bootstrap.sh`	Idempotent DOKS workload: operator, pull secret, object-store secret, RayCluster (needs existing kubeconfig)
`drtoller cloud status`	`cloud-status.sh`	kubectl context, operator, ray pods/svc, `.env` `RAY_ADDRESS`
`drtoller ray attach`	`cloud-ray-attach.sh`	Head LB IP → `.env` `RAY_ADDRESS` + recreate Airflow; then `ray status`

AWS/EKS, GCP/GKE, Vertex/Anyscale installers are backlog — not in this CLI yet. Current multi-host path is DigitalOcean DOKS + compose on the VM.

`drtoller up` sequence

Ensure Docker daemon
ensure-env.sh → .env
Create DATADIR subtrees + Postgres data dir permissions
fix-artifact-perms.sh
docker compose pull (skip local Ray images)
docker compose build ray-head (+ best-effort ray-worker via profile build-ray-worker for KubeRay)
Postgres + Redis → migrate-postgres.sh
MinIO → minio-init → Qdrant → airflow-init
Ray head, Prometheus, Grafana, Airflow, Streamlit
sync-grafana-dashboards.sh → restart Grafana

First run downloads large images; expect several minutes.

After deploy / git pull

cd /opt/drtoller.platform/stack
drtoller update
drtoller verify

drtoller update does not rebuild the Airflow image. It:

runs Postgres migrate (migrate-postgres.sh: codegen, rename-to-drt_* prefixes, framework procs, step DDL, drop orphans vs pg_object_inventory.txt);
regenerates Grafana JSON and restarts Grafana;
rebuilds ray-head (and best-effort ray-worker for KubeRay) and force-recreates Airflow services + Ray head;
restarts Streamlit and Prometheus.

Ray cluster health: drtoller ray status (Framework) or, for DOKS, drtoller verify after drtoller ray attach (see Ray).

Multi-host Ray on DOKS (stage 1)

Prerequisites: compose stack up (drtoller up), DOKS Ready, KUBECONFIG, .env with MINIO_API_BIND=<VM VPC IP>.

export KUBECONFIG=~/.kube/drtoller-framework-dev.yaml
drtoller cloud bootstrap --provider digitalocean --attach
drtoller verify

Split form: drtoller cloud bootstrap --provider digitalocean then drtoller ray attach.
Internals: Platform stack/kuberay/ONTO_RUNBOOK.md.

Postgres object names must use drt_tbl_ / drt_proc_ — see PostgreSQL naming.

GitHub Actions deploy (.github/workflows/deploy.yml) can trigger variants via commit tags: [drtupdate], [drtrestart], [drtrestart scheduler], [skip deploy].

Environment updates

Situation	Command
New keys in `env.example` after upgrade	`drtoller env update --merge`
Reset `.env` to template (secrets backup in `.env.bak`)	`drtoller env update`
Changed pip requirements	edit `.env` → `drtoller env update` or force-recreate Airflow
CRLF in `.env` (`$'\r': command not found`)	`drtoller env update` or `tr -d '\r' < .env`

Partial up / down

drtoller down stop ray-head airflow-worker
drtoller up only airflow-worker ray-head
drtoller restart grafana

Day-1 bootstrap (once)

After first successful drtoller up:

cd /opt/drtoller.platform/stack
./bin/day1-bootstrap.sh

Health check

drtoller verify

Reboot

Because restart: "no", reboot does not auto-start the stack. Run drtoller up after server boot (or add your own systemd unit).