Local ML Studies

Build and evaluate classic machine-learning classifiers with immutable algorithm trials, trusted metrics, reproducible provenance, and a one-shot held-out test. This guide follows a real Polymarket large-move Study from public data collection to its final receipt.

Use a Study when the job is a structured prediction problem rather than a generative behavior problem. A bag-of-words spam classifier, a gradient-boosted tabular model, or a numeric time-series baseline may be faster, cheaper, and easier to evaluate than fine-tuning a transformer. TT Local Studies make the algorithm an explicit experiment alongside the data.

The first built-in Study path supports binary classification with a numeric logistic-regression baseline. It runs on CPU, including on an NVIDIA DGX Spark; the GPU is not required for this example.

The Study loop

1. Define a task, feature allowlist, target semantics, metric, and fixed train/validation/test splits in a StudySpec.
2. Lock the benchmark so data, schema, split identities, and temporal boundaries cannot drift silently.
3. Run immutable TrialSpecs. The runner sees training labels but receives validation rows without their target.
4. Let a person or AI create new trial IDs and adjust algorithms or parameters using validation results only.
5. Promote one fitted candidate after its saved model exactly replays the selected validation predictions.
6. Consume the held-out test once and keep its durable receipt as the final evidence.

Prerequisites

• Node.js 22 or newer
• uv for the locked Python runner environment
• Python 3 for the example's standard-library preparation script
• Enough disk space for your source snapshots and generated splits

First check whether your installed release contains the complete Study workflow:

npm install -g @tuned-tensor/local@latest
tt-local studies test --help

If that command is unavailable, install current main from a packed source checkout. This also gives you the runnable example:

git clone https://github.com/tunedtensor/tuned-tensor-local.git
cd tuned-tensor-local
npm ci
npm pack
npm install -g ./tuned-tensor-local-*.tgz

curl -LsSf https://astral.sh/uv/install.sh | sh
tt-local --version
uv --version

cd examples/polymarket-anomaly-study

The example directory contains the StudySpec, one TrialSpec, and the deterministic split-preparation script. Generated data, locks, trial artifacts, candidates, and test receipts are intentionally not committed.

1. Prepare a labeled source CSV

Start with timestamped market snapshots from a source you are authorized to use. The file should span at least 25 hours because the target looks one hour into the future and the benchmark needs enough history on both sides of two purged split boundaries.

Save the file as data/labeled.csv. It must contain:

• A unique id for every row.
• observed_at and future_observed_at RFC 3339 UTC timestamps.
• has_two_sided_book and mid fields for quality filtering.
• The six numeric model inputs declared later in the StudySpec.
• A big_move label containing 0 or 1.

2. Define the future label

Here, big_move=1 means the outcome midpoint moved by at least 500 basis points—an absolute 5 percentage-point move—within the next hour. Compute labels from later observations, and restrict model inputs to values available at observed_at. Labels should not be generated by an LLM.

The reference CSV contained 82,837 labeled rows: 81,007 negatives and 1,830 positives. Average precision is therefore the primary metric; it is more informative than accuracy for this imbalanced event.

3. Prepare purged train, validation, and test data

Run the supplied preparation script with boundaries inside your source time range:

python3 prepare_splits.py data/labeled.csv data \
  --training-end 2026-07-03T23:00:00Z \
  --validation-start 2026-07-04T00:06:00Z \
  --validation-end 2026-07-04T08:00:00Z \
  --test-start 2026-07-04T09:06:00Z

The two gaps are 66 minutes. Each exceeds the declared 60-minute label horizon plus the 5-minute embargo, preventing a row's future label window from crossing into the next split. The script filters to two-sided books with midpoints between 0.02 and 0.98, preserves missing numeric values for the imputer, samples deterministically by row ID, and sorts selected rows chronologically.

4. Understand the StudySpec

polymarket.study.json declares the problem independently of any algorithm:

{
  "schema_version": 1,
  "name": "Polymarket LP large-move risk",
  "task": {
    "type": "binary_classification",
    "id_column": "id",
    "input_columns": [
      "spread_bps",
      "liquidity",
      "volume_24h",
      "bid_depth_3c",
      "ask_depth_3c",
      "imbalance_3c"
    ],
    "target_column": "big_move",
    "labels": { "negative": "0", "positive": "1" },
    "primary_metric": "average_precision"
  },
  "dataset": {
    "format": "csv",
    "splits": {
      "training": "data/training.csv",
      "validation": "data/validation.csv",
      "test": "data/test.csv"
    },
    "temporal": {
      "policy": "ordered_purged",
      "event_time_column": "observed_at",
      "label_end_time_column": "future_observed_at",
      "label_horizon_seconds": 3600,
      "embargo_seconds": 300
    }
  }
}

The feature allowlist is a leakage boundary. Future timestamps and the target remain in trusted benchmark data for certification and scoring, but they are not model inputs.

5. Lock the benchmark

tt-local studies lock polymarket.study.json
tt-local studies validate polymarket.study.json

The lock records exact source hashes and sizes, columns, row counts, disjoint IDs, label presence, task semantics, and observed temporal ranges. studies validate is read-only. Any later data or StudySpec drift fails before a trial starts.

6. Run an algorithm trial

The included TrialSpec selects the bundled numeric runner and makes its parameters explicit:

{
  "schema_version": 1,
  "id": "polymarket-logreg-balanced-c1-v1",
  "name": "Balanced numeric logistic regression (C=1)",
  "runner": {
    "builtin": "numeric_logistic_regression",
    "timeout_ms": 300000
  },
  "parameters": {
    "c": 1,
    "class_weight": "balanced",
    "max_iter": 1000,
    "random_seed": 42
  }
}

tt-local studies run \
  polymarket.study.json \
  logreg-balanced-c1.trial.json

The runner fits median imputation, missingness indicators, feature scaling, and logistic regression. It receives training labels, but the validation protocol contains only IDs and allowlisted inputs. TT Local joins returned probabilities to trusted labels and computes the metrics.

The Spark run returned:

{
  "primary_metric": "average_precision",
  "primary_score": 0.28062119335103797,
  "metrics": {
    "average_precision": 0.28062119335103797,
    "roc_auc": 0.8732770668539412,
    "f1_at_0_5": 0.20065789473684212
  },
  "prediction_count": 5000,
  "duration_ms": 1503
}

7. Let AI adjust the algorithm safely

Give an AI agent the StudySpec, benchmark lock, trial report, and artifact paths—not the held-out test labels. It can propose a new TrialSpec with a new immutable id, change c, class weighting, convergence limits, or random seed, run it, and compare validation reports. Record the reason for each change in version control or your experiment log.

Never overwrite or reuse a trial ID. Never choose a candidate from test performance. Stop iterating when improvements are small, unstable, or no longer justified by complexity.

Command-backed runners already let you validate other algorithms against the same label-blind protocol. In the current release, replay-verified promotion and one-shot testing require the bundled numeric logistic-regression runner. A reusable saved-model contract for more model families is the next extension point.

8. Promote the selected candidate

tt-local studies promote \
  polymarket.study.json \
  logreg-balanced-c1.trial.json

Promotion copies the fitted model, runner source, dependency lock, implementation manifest, validation projection, and predictions into a write-once candidate directory. It reloads the saved model and rejects promotion unless all 5,000 replayed probabilities reproduce the selected validation predictions within 1e-12.

The reference run replayed all 5,000 rows with maximum absolute difference 0.

9. Consume the held-out test once

tt-local studies test polymarket.study.json

Before touching the test file, TT Local verifies the Study, benchmark lock, candidate, model, frozen predictor, dependency lock, and a fresh label-free validation replay. It then atomically claims the held-out content and target semantics in the global TT_LOCAL_HOME ledger.

Only IDs and allowlisted features reach the frozen predictor. TT Local retains labels in its trusted scoring process and writes the receipt last:

{
  "status": "succeeded",
  "evaluation": {
    "primary_metric": "average_precision",
    "primary_score": 0.1679596802280865,
    "metrics": {
      "average_precision": 0.1679596802280865,
      "roc_auc": 0.813649167501233,
      "f1_at_0_5": 0.0927960927960928
    },
    "prediction_count": 5000
  },
  "execution": {
    "duration_ms": 540,
    "runtime": {
      "evidence": {
        "runtime": {
          "python": "3.12.3",
          "numpy": "2.5.1",
          "scikit_learn": "1.9.0",
          "joblib": "1.5.3"
        }
      }
    }
  }
}

The lower test average precision is a useful result, not a failure of the workflow. The test period had a lower positive rate and the model generalized less strongly than it did on validation. That is exactly why the test must remain outside the optimization loop.

A repeat is rejected:

Held-out test identity 942955... has already been consumed;
successful, failed, incomplete, and crashed claims are all one-shot

Where the evidence lives

• polymarket.study.lock.json — immutable benchmark evidence
• .tt-local/study-trials/<trial-id>/ — projected data, logs, predictions, fitted model, implementation snapshot, and validation report
• .tt-local/study-candidates/<study-file>/ — frozen candidate and replay evidence
• $TT_LOCAL_HOME/study-test-claims/v1/<claim-id>/receipt.json — final held-out metrics, producer/runtime versions, hashes, and generated artifact references

Current scope and next steps

Studies now provide the reliable inner loop: fixed data contracts, algorithm trials, validation feedback, provenance, candidate freezing, and one-shot testing. They do not yet autonomously search Hugging Face for datasets, create labels, choose between all model families, or orchestrate a model search without an external agent.

The clean path forward is incremental: add reusable saved-model contracts for more classic and time-series runners; add dataset and labeling adapters that emit a StudySpec plus auditable label evidence; add comparison and stopping policies for AI-driven trial generation; then add higher-level orchestration. The benchmark and one-shot test boundaries should remain stable while those capabilities expand.