--- name: tuned-tensor description: Use when an agent needs to fine-tune and evaluate small open-weight models with TT Local or the optional managed Tuned Tensor service. --- # Tuned Tensor Tuned Tensor turns a behaviour spec into a fine-tuned small open-weight model and a paired, inspectable report comparing it with the run baseline on representative cases. Use `tt-local` from `@tuned-tensor/local` for local CUDA or DGX Spark runs on Linux; use `tt` for the optional managed service. ## Local Quick Start ```bash npm install -g @tuned-tensor/local tt-local init --name "Customer Support Bot" --model Qwen/Qwen3.5-2B --profile spark # Edit tunedtensor.json, then: tt-local doctor tunedtensor.json --config local-runner.json tt-local run tunedtensor.json --config local-runner.json tt-local runs report --config local-runner.json ``` See `/docs/local-training` for configuration, evaluation, and artifact details. ## Managed Quick Start Install the CLI: ```bash npm install -g @tuned-tensor/cli ``` Authenticate with an API key from the Tuned Tensor dashboard: ```bash tt auth login tt auth status ``` Create a local behaviour spec: ```bash tt init --name "Customer Support Bot" --model Qwen/Qwen3.5-2B ``` Edit `tunedtensor.json`, then validate and sync it: ```bash tt eval tt push ``` Start and watch a training run: ```bash tt runs estimate tt runs start tt runs diagnose tt runs watch tt runs report ``` Inspect, download, or locally serve the resulting model: ```bash tt models list tt models get tt models download --output model.tar.gz tt models setup-runtime tt models serve --spec tunedtensor.json ``` ## Managed Agent Workflow 1. Check authentication with `tt auth status`. 2. If no local spec exists, run `tt init` with the nearest matching supported base model. 3. Treat `tunedtensor.json` as the source of truth. Edit it directly, preserve its `id` when updating an existing spec, and commit meaningful spec changes. 4. Run `tt eval` before `tt push`. 5. Run `tt push` to create or update the remote behaviour spec. 6. Start a run with `tt runs start `, inspect live learning progress with `tt runs diagnose `, and poll with `tt runs watch `. 7. Inspect results with `tt runs get ` and use `tt runs report ` for side-by-side Expected, Base, and Tuned outputs before starting another iteration. 8. Download completed models with `tt models download` only when the user needs a local artifact. 9. Use `tt models setup-runtime` once before local reference serving, then `tt models serve --spec tunedtensor.json` to serve a model with its behaviour spec prompt applied. ## Common Commands ```bash tt specs list tt specs get tt runs list tt runs get tt runs estimate tt runs diagnose tt runs report tt datasets upload training.jsonl --name "Training data" tt datasets list tt label upload unlabeled.jsonl --spec --name "Labeling job" tt label watch tt label rows --status labeled tt label accept --all tt label promote --name "Reviewed labels" tt models setup-runtime tt models serve --spec tunedtensor.json tt balance tt topup --amount 25 ``` Use `--json` when another program needs to parse command output. ```bash tt --json runs get tt --json runs report ``` ## Monitoring Runs After `tt runs start `, capture the returned run ID and monitor the job until it reaches a terminal status. Use `tt runs diagnose ` for the most useful live view: it surfaces the current stage, recent events, evaluation progress, and any actionable failure details. Use `tt runs watch ` when you want a simple polling loop, `tt runs get ` for the latest persisted status, and `tt runs report ` to inspect aggregate deltas plus side-by-side Expected, Base, and Tuned outputs. For failed-case triage, use `tt runs report --mode failures`; for held-out results, use `--split test` or `--split all`. ## Preflight Estimates Use `tt runs estimate` to preview rough run duration, cost, and whether the run is expected to use free monthly quota or paid credits before starting. The dashboard and REST API expose the same preflight estimate. ```bash tt runs estimate --epochs 4 tt runs estimate --dataset --train-ratio 0.8 ``` REST equivalent: ```bash curl -X POST https://tunedtensor.com/api/v1/behavior-specs//runs/estimate \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d '{"augment":true,"hyperparameters":{"n_epochs":4}}' ``` The response includes estimated training tokens, estimated cost, and a duration range with confidence/sample count. Duration is historical and approximate; final cost still reconciles from provider-reported training tokens. ## Teacher Labeling Use `tt label` when the user has unlabeled rows and wants Tuned Tensor to draft outputs under an existing behaviour spec. JSONL inputs should contain one `{"input": "..."}` object per line. CSV inputs require `--input-column`. ```bash tt label upload unlabeled.jsonl --spec --name "Support labels" --watch tt label rows --status labeled tt label accept --all tt label promote --name "Reviewed support labels" ``` Labeling jobs run deterministic sanitization before teacher calls. Ordinary PII such as email addresses, phone numbers, SSNs, and credit-card-like values is replaced with redaction placeholders. Rows containing secret-like content, including password assignments, bearer tokens, API keys, connection strings, or private keys, are marked failed and are not sent to the teacher model. Promotion re-scans reviewed inputs and outputs before writing the dataset. Blocked rows are excluded, and promotion fails if reviewer edits introduce secret-like content. Do not print sensitive source rows or teacher outputs while inspecting labeling jobs; summarize row counts, statuses, sanitization status, and finding kinds instead. Use a custom API base URL only for local or staging environments: ```bash tt -u https://your-api.example.com specs list ``` ## Local Model Serving Use the CLI for local reference serving when a user wants to test a completed model locally: ```bash tt models setup-runtime tt models serve --spec tunedtensor.json ``` `tt models serve` accepts a model ID, a downloaded `.tar.gz` artifact, or an extracted Hugging Face model directory. It exposes an OpenAI-compatible local endpoint at `/v1/chat/completions` and a `/health` endpoint. By default it applies the compiled behaviour spec prompt from `tunedtensor.json`, so responses preserve the behaviour spec prompt used during training. Useful options: ```bash tt models serve --spec tunedtensor.json --device auto tt models serve --spec tunedtensor.json --device mps tt models serve --spec tunedtensor.json --device cuda tt models serve model.tar.gz --spec tunedtensor.json tt models serve ./models/my-model --spec tunedtensor.json ``` Use `--no-spec-prompt` only when intentionally testing raw model behaviour. ## Managed Base Models - `google/gemma-4-E2B-it` - `google/gemma-4-E4B-it` - `Qwen/Qwen3.5-2B` - `Qwen/Qwen3-VL-2B-Instruct` - `Qwen/Qwen3.5-4B` - `meta-llama/Llama-3.2-3B-Instruct` - `microsoft/Phi-4-mini-instruct` - `ibm-granite/granite-3.3-2b-instruct` - `bigcode/starcoder2-3b` ## REST API Fallback Base URL: ```text https://tunedtensor.com/api/v1 ``` Authenticate REST calls with: ```text Authorization: Bearer ``` Prefer these docs before constructing raw requests: - `/docs/quickstart` - `/docs/cli` - `/docs/authentication` - `/docs/behavior-specs` - `/docs/runs` - `/docs/datasets` - `/docs/models` - `/docs/billing` ## Safety Rules - Never commit API keys, downloaded model artifacts, `.env` files, or credentials. - Never commit local serving runtimes, extracted model directories, or model cache artifacts. - Do not print full API keys in logs or final answers. - Check `tt balance` before starting expensive or repeated runs. - If a command fails with `402 insufficient_credits`, run `tt topup` or ask the user to add credits before retrying. - Do not delete remote specs, datasets, runs, or models unless the user explicitly asks.