Tuned Tensor

Local Training

Run Tuned Tensor-style fine-tuning on hardware you control with tt-local. Specs, datasets, model artifacts, run events, reports, and the local metadata index stay on local disk.

Use this when you want cloud-independent SFT, DPO, staged runs, local baseline-vs-candidate evaluation, or iterative training from a previous local adapter. For serving a completed hosted model artifact locally, use tt models serve.

Get Started

Install the latest local runner and verify that both tt-local and uv are available. Use tt-local 0.2.7 or newer for explicit base-model prefetch, packaged bundled training, DPO, staged runs, command backends, and continued fine-tuning.

npm install -g @tuned-tensor/local@latest
tt-local info
uv --version

If uv is missing:

curl -LsSf https://astral.sh/uv/install.sh | sh

Create a Project Folder

Start each local training job in its own folder. That folder holds your behavior spec, runner config, local artifacts, and store. tt-local init creates the first spec file, tunedtensor.json, in the current directory.

mkdir support-bot-local
cd support-bot-local
tt-local init --name "Support Bot" --model Qwen/Qwen3.5-2B

Open tunedtensor.json and replace the placeholder system prompt, guidelines, and examples with the behavior you want the model to learn. For field-by-field guidance, including local-only keys such as training_method and dataset_prebuilt, see the spec file guide.

Dry Run First

A dry run validates the spec, dataset conversion, stage orchestration, SQLite-backed store, and report writing without downloading a model or starting GPU training.

cat > local-runner.json <<'JSON'
{
  "artifactRoot": ".tt-local/artifacts",
  "storeRoot": ".tt-local/store",
  "dryRun": true,
  "evaluation": {
    "inference": {
      "provider": "none"
    },
    "scoring": {
      "mode": "exact_match"
    },
    "maxExamples": 2
  }
}
JSON

tt-local doctor --config local-runner.json
tt-local validate --config local-runner.json
tt-local run --config local-runner.json
tt-local runs report <run-id> --config local-runner.json

Real Training Config

For real bundled uv training, install tt-local, use durable artifact and store paths, set dryRun to false, and keep the bundled uv project pointed at training/local-runner. In tt-local 0.2.6 and newer, that Python project is included in the npm package and resolves relative to the installed CLI, so a local source checkout is not required. The selected training script is chosen from training_method.

Save this as spark-runner.json:

{
  "artifactRoot": "/mnt/tt-local/artifacts",
  "storeRoot": "/mnt/tt-local/store",
  "dryRun": false,
  "training": {
    "backend": "uv",
    "project": "training/local-runner"
  },
  "evaluation": {
    "inference": {
      "provider": "transformers",
      "project": "training/local-runner",
      "script": "training/local-runner/src/evaluate.py",
      "maxNewTokens": 256,
      "temperature": 0,
      "topP": 1
    },
    "scoring": {
      "mode": "llm_judge",
      "fallback": "exact_match"
    },
    "maxExamples": 20
  },
  "paths": {
    "modelCache": "/mnt/tt-local/huggingface-cache"
  }
}
export OPENROUTER_API_KEY="your_openrouter_key"

tt-local doctor --config spark-runner.json
tt-local validate --config spark-runner.json
tt-local models prefetch tunedtensor.json --config spark-runner.json
tt-local run --config spark-runner.json --verbose
tt-local runs watch <run-id> --config spark-runner.json
tt-local runs report <run-id> --config spark-runner.json

tt-local models prefetch downloads the base model into paths.modelCache before the first real run. If you skip it,tt-local run still downloads the model automatically when baseline evaluation or training first loads it. tt-local run writes live stage updates to stderr and final structured JSON to stdout. Use --verbose for subprocess logs or --quiet for JSON-only automation.

Custom Training Files

The bundled paths are only special-cased for the default runner. To use your own uv project or script, set training.project, training.script, evaluation.inference.project, or evaluation.inference.script to your own paths; custom values stay relative to the directory where you run tt-local. For a fully custom trainer, set training.backend to "command" and provide training.command.

{
  "training": {
    "backend": "uv",
    "project": "./my-runner",
    "script": "./my-runner/train_adapter.py"
  }
}

SFT

Supervised fine-tuning is the default path. You can train from examples in tunedtensor.json or point at a prebuilt chat JSONL dataset.

{
  "name": "Email triage",
  "base_model": "Qwen/Qwen3.5-2B",
  "system_prompt": "Return a compact triage decision.",
  "examples": [
    {
      "input": "Subject: build failed\nBody: CI is red on main.",
      "output": "{\"triage\":\"engineering\",\"priority\":\"high\"}"
    }
  ],
  "hyperparameters": {
    "n_epochs": 3,
    "batch_size": 1,
    "gradient_accumulation_steps": 8,
    "lora_rank": 16,
    "save_adapter_only": true
  }
}

Prebuilt SFT datasets use chat JSONL rows with messages. Provide held-out test or validation data when possible so the report does not score on the training split.

{
  "dataset_prebuilt": {
    "training": "file:///data/email-train.chat.jsonl",
    "validation": "file:///data/email-validation.chat.jsonl",
    "format": "chat_jsonl"
  }
}

Multimodal SFT

For document/OCR or image-to-JSON workflows, use a supported image-text model such as Qwen/Qwen3-VL-2B-Instruct. Image values in JSONL may be absolute paths, paths relative to the JSONL file, file:// URIs, HTTP(S) URLs, or data: URIs.

Thinking-Mode Models

If a model opens hidden thinking and consumes the output budget, pass matching chat template kwargs to both training and evaluation.

In spark-runner.json:

{
  "evaluation": {
    "inference": {
      "chatTemplateKwargs": { "enable_thinking": false }
    }
  }
}

In tunedtensor.json:

{
  "hyperparameters": {
    "chat_template_kwargs": { "enable_thinking": false }
  }
}

DPO

Offline DPO trains a text causal-LM model from preference rows. Set training_method to dpo and provide a prebuilt preference JSONL training file.

{
  "name": "Preference trainer",
  "training_method": "dpo",
  "base_model": "Qwen/Qwen3.5-2B",
  "dataset_prebuilt": {
    "training": "file:///data/preferences.jsonl",
    "test": "file:///data/dpo-eval.chat.jsonl",
    "format": "preference_jsonl"
  },
  "hyperparameters": {
    "n_epochs": 1,
    "dpo_beta": 0.1,
    "dpo_loss_type": "sigmoid",
    "max_prompt_length": 512,
    "max_completion_length": 256
  }
}

Each preference row must contain string fields:

{"prompt":"Summarize status: build passed.","chosen":"Build passed.","rejected":"The build failed."}

The bundled DPO trainer is text-only. Use training.backend: "command" for custom DPO trainers or non-standard model families.

Continued Fine-Tuning

A continued run starts from a previously fine-tuned TT Local PEFT adapter, evaluates that parent adapter as the baseline, and then trains a new child adapter from it. The parent and child must use the same base model.

tt-local models list --config spark-runner.json
tt-local run tunedtensor.json \
  --config spark-runner.json \
  --parent-model local-<previous-run-id>

You can also pass the parent artifact directly, or store it in the spec for config-only workflows.

tt-local run tunedtensor.json \
  --config spark-runner.json \
  --parent-model-artifact file:///path/to/parent-adapter-or-model.tar.gz
{
  "hyperparameters": {
    "parent_model_artifact": "file:///path/to/parent-adapter-or-model.tar.gz"
  }
}

Staged and Resumable Runs

Stages let you prepare, evaluate, train, score, or report without repeating work. Existing stage artifacts are reused by default; pass --force to recompute.

tt-local run tunedtensor.json --config spark-runner.json --stage prepare --run-id <run-id>
tt-local run tunedtensor.json --config spark-runner.json --stage baseline --run-id <run-id>
tt-local run tunedtensor.json --config spark-runner.json --stage train --run-id <run-id>
tt-local run tunedtensor.json --config spark-runner.json --stage candidate --run-id <run-id>
tt-local run tunedtensor.json --config spark-runner.json --stage score --run-id <run-id>
tt-local run tunedtensor.json --config spark-runner.json --stage report --run-id <run-id>

Use --model-artifact with the candidate stage to evaluate an externally produced adapter against the same prepared baseline.

tt-local run tunedtensor.json \
  --config spark-runner.json \
  --stage candidate \
  --run-id <run-id> \
  --model-artifact file:///path/to/external-adapter

Evaluation and Scoring

  • evaluation.maxExamples caps scored examples. If unset, max_eval_examples from the run request is used.
  • Capped evaluation uses deterministic sampling, not a prefix. Pin it with evaluation.sampleSeed.
  • Spec-only SFT runs hold out about 20% of examples for evaluation when possible.
  • Prebuilt SFT datasets evaluate on dataset_prebuilt.test, then dataset_prebuilt.validation. Training-split evaluation requires evaluation.allowPrebuiltTrainingEval: true.
  • Baseline evals are cached under <storeRoot>/cache/baseline-evals/. Disable with evaluation.baselineCache: false.

JSON Field Scoring

{
  "evaluation": {
    "scoring": {
      "mode": "json_fields",
      "fields": ["triage", "priority", "should_process"]
    }
  }
}

If fields is omitted, every key in the expected JSON object is scored. Missing expected fields count as incorrect.

Comparing Runs

tt-local runs compare <run-id-a> <run-id-b> --config spark-runner.json

Comparisons align shared prompts, show headline score deltas, and separate shared-example movement from new-example effects.

Command Backends

Use command backends for custom trainers, external model families, or non-Hugging Face artifact layouts. Command trainers receive the run hyperparameters through TT_HYPERPARAMETERS_PATH and local SageMaker-style paths such as SM_CHANNEL_TRAINING and SM_MODEL_DIR.

In the spec, use an external or command-backed base model id:

{
  "name": "Custom trainer",
  "base_model": "external:karpathy/nanochat",
  "examples": [
    { "input": "hello", "output": "greeting" }
  ]
}

In the runner config, provide the custom trainer and evaluator:

{
  "training": {
    "backend": "command",
    "command": ["python", "train_adapter.py"],
    "artifact": {
      "framework": "nanochat",
      "format": "custom-directory",
      "entrypoint": "batch_command",
      "servable": false
    }
  },
  "evaluation": {
    "inference": {
      "provider": "batch_command",
      "command": ["python", "eval_adapter.py"]
    }
  }
}

Teacher Labeling

tt-local labelturns unlabeled JSONL or CSV rows into training examples with an OpenRouter teacher model under the spec's system message.

export OPENROUTER_API_KEY=...
tt-local label rows.jsonl --output labeled.jsonl
tt-local label rows.csv --input-column text --model openai/gpt-5.5

JSONL rows use { "input": "..." }. Rows with output are skipped. Use --spec to choose the behaviour spec and --dry-run to parse, sanitize, and estimate without calling the teacher.

Secret-like content blocks a row. PII such as emails, phone numbers, SSNs, and card numbers is redacted before teacher calls. Outputs are written under <artifactRoot>/labeling/<job-id>/.

Files and Store

If storeRoot is omitted, tt-local uses TT_LOCAL_HOME or ~/.tuned-tensor-local. The SQLite index is rebuildable from canonical JSON files.

  • <storeRoot>/metadata.sqlite: local index for specs, runs, models, and event listings.
  • <storeRoot>/specs/<spec-id>/spec.json: canonical behaviour spec snapshot.
  • <storeRoot>/runs/<run-id>/state.json and request.json: canonical run state and request snapshot.
  • <storeRoot>/runs/<run-id>/progress.jsonl: stage and training progress events.
  • stage-metadata.json, training-report.json,baseline-eval.json, candidate-eval.json, and run-report.json: staged workflow artifacts under the run artifact directory.
  • training/training.log: uv or command fine-tuning output under the run artifact directory.
tt-local store rebuild-index --config spark-runner.json

Commands

tt-local init --name "Support Bot" --model Qwen/Qwen3.5-2B
tt-local doctor --config spark-runner.json
tt-local validate --config spark-runner.json
tt-local models prefetch tunedtensor.json --config spark-runner.json
tt-local run --config spark-runner.json --verbose
tt-local label rows.jsonl --output labeled.jsonl
tt-local runs list --config spark-runner.json
tt-local runs get <run-id> --config spark-runner.json
tt-local runs events <run-id> --config spark-runner.json
tt-local runs watch <run-id> --config spark-runner.json
tt-local runs report <run-id> --config spark-runner.json
tt-local runs compare <run-id-a> <run-id-b> --config spark-runner.json
tt-local models list --config spark-runner.json
tt-local models get <model-id> --config spark-runner.json
tt-local store rebuild-index --config spark-runner.json
tt-local serve --config spark-runner.json

DGX Spark

On a Spark or other GPU host, keep the Hugging Face cache and artifacts on persistent local storage before starting long runs.

nvidia-smi
uv --version
uv run python --version

git clone https://github.com/tunedtensor/tuned-tensor-local.git
cd tuned-tensor-local
npm install
npm run build
npm link
tt-local doctor --config examples/local-runner.json

Set paths.modelCache to a persistent Spark-local cache, and keep artifactRoot and storeRoot on a volume that will survive shell sessions.