Tuned Tensor

Local Training

Fine-tune, evaluate, verify, and serve a model on your own NVIDIA GPU. Datasets, run state, reports, and model artifacts stay on the execution host.

Supported path

The current local workflow supports:

  • Text supervised fine-tuning (SFT) with LoRA/PEFT on CUDA
  • Qwen/Qwen3.5-2B (pinned to snapshot 15852e8c16360a2fea060d615a32b45270f8a8fc)
  • nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 on DGX Spark-class unified memory
  • meta-models/Muse-Glimmer-30B (vision-language: text tower trained, vision tower frozen)
  • Exact-match or selected JSON-field evaluation
  • OpenAI-compatible local serving

The Nemotron path uses activation checkpointing and adapts bounded shared attention/Mamba projections rather than every routed expert matrix. Evaluation and serving may use CPU.

Requirements

  • Node.js 22.19+
  • uv
  • A Linux host with a compatible NVIDIA GPU
  • Enough disk space for the model cache and run artifacts
curl -fsSL https://tunedtensor.com/install.sh | sh
tt --version
tt status
uv --version
nvidia-smi

Create a project

mkdir support-adapter
cd support-adapter
tt init \
  --name "Support Adapter" \
  --model Qwen/Qwen3.5-2B \
  --profile spark

This creates tunedtensor.json and local-runner.json. Replace both placeholder examples with different, representative examples before running.

Preflight and run

tt doctor tunedtensor.json
tt validate tunedtensor.json
tt models prefetch tunedtensor.json
tt models verify-base tunedtensor.json
tt run tunedtensor.json

Commands automatically discover local-runner.json beside the spec. Use --config /path/to/local-runner.json to select another configuration.

Inspect and verify

tt runs list
tt runs get <run-id>
tt runs events <run-id>
tt runs report <run-id>
tt models verify local-<run-id>

The report compares the base and tuned model on the same held-out prompts. It is evidence on those examples, not a guarantee of general improvement.

Use a chat JSONL dataset

For larger datasets, add dataset_prebuilt to tunedtensor.json:

{
  "dataset_prebuilt": {
    "training": "data/train.jsonl",
    "validation": "data/validation.jsonl",
    "format": "chat_jsonl"
  }
}

Each line contains a messages array. The final message is the assistant answer; earlier messages form the prompt. Use a distinct validation or test file for held-out evaluation.

Protect general capability

Add a separate regression suite to local-runner.json when activation should be blocked by a drop in broad capability:

{
  "evaluation": {
    "generalRegression": {
      "dataset": "evals/general.jsonl",
      "maxScoreDrop": 0.03,
      "maxPassRateDrop": 0.05
    }
  }
}

Activate and serve

tt models activate local-<run-id>
tt models active
tt serve active --config local-runner.json

# Restore the previous adapter or protected base
tt models rollback

Activation requires a completed run, a verified model artifact, and a passing general-regression result. Without that suite, tt models activate fails closed. tt serve active fails if nothing is activated; it does not silently serve the protected base model. Serve a specific adapter with tt models serve local-<run-id> until then.

Serve a specific verified adapter:

tt models serve local-<run-id> \
  --spec tunedtensor.json \
  --host 127.0.0.1 \
  --port 8000

The default bind is localhost. A non-loopback bind requires --allow-remote and an API key supplied through --api-key-env. tt serve base does not automatically inject the adjacent project spec; pass --spec tunedtensor.json when the server should enforce its instructions.

Next steps