The KOTH Protocol

King-of-the-Hill over miner-run, TEE-attested benchmarks. Miners run the owner's benchmark suite inside their own confidential VM and publish an attested proof to their own HuggingFace repo. Validators only verify the proof and grade the reports — no validator-side inference, no centralized API. A formula over the verified reports crowns the king.

The load-bearing idea: because the artifact — source and weights — is public and cryptographically bound to what ran, cheating is publicly detectable rather than something we must prevent by hiding data. That is what makes static, public benchmarks safe here.

The trust inversion

Other benchmark-scored subnets run evaluation on infrastructure you must trust. KOTH moves execution into the miner's own TEE and keeps it trustworthy with hardware attestation.

The enclave runs the miner's arbitrary agent but hands it only a metered call_model, and it stamps the agent's source_hash/weights_hash into the attestation. So anyone — not just a validator — can check that a given public artifact produced a given score, without re-running it.

Data models

BenchmarkResult:                      # one graded item
  benchmark    str                    # "math" | "mmlu" | "gpqa" | "swe"
  task_id      str                    # validator re-derives the gold from the nonce
  answer       str                    # graded validator-side against PUBLIC gold
  cost_usd     float                  # metered by the runtime — un-forgeable

Proof:                                # one per miner-epoch; the unit validators verify
  epoch, nonce, hotkey                # chain-derived nonce; hotkey bound (anti-resubmit)
  source_hash, weights_hash, model_id # BINDING: ties the score to the public artifact
  results: [BenchmarkResult]          # the whole suite in one attested run
  total_cost_usd, n_calls, call_log_hash
  measurement                         # runtime image hash; must be on the approved list
  quote        = Platform.quote(measurement, report_data)   # hardware signature
  report_data()= sha256(payload-minus-quote)                # binds EVERY field above

The on-chain commit is salted with your hotkey:

koth1|<hf_repo>|<revision>|sha256(source_hash|weights_hash|hotkey)

One hardware signature over {measurement, report_data} asserts two things at once: "this approved enclave ran" and "it produced exactly this payload." Mutate any field and report_data() changes, breaking the quote.

Miner flow

1. Build the agent. A Python source file defining build_agent(weights) -> agent, where agent(prompt, call_model) -> answer. call_model is your only channel to the pool.
2. Run the suite in your TEE. The measured runtime samples the epoch's nonce-seeded slice, executes your agent under no-egress confinement, meters every pool call, and produces an attested Proof plus the full behavioural trace.
3. Publish the bundle to your own HF repo: source.py, weights.bin, proofs/<epoch>.json, traces/<epoch>.json. All public, all auditable.
4. Commit on-chain via set_reveal_commitment. The reveal delay defeats front-running; the hotkey salt defeats commit-copying.

Validator flow

Cheap, and re-execution-free. Per epoch, for each committed miner:

1. Download the public bundle yourself, recompute source_hash/weights_hash, and verify_commit them against the on-chain commit. The binding is checked here — never taken on the miner's word.
2. verify_proof against those recomputed hashes: quote valid → measurement approved → report_data bound → issued (epoch, nonce) → bound hotkey → hashes match.
3. Re-derive the same nonce-seeded slice and grade the attested answers against the public gold. Missing or substituted tasks count as wrong. No agent re-run.
4. Check the trace hashes to call_log_hash, and that every scored task has ≥ 1 metered pool call.
5. Scan the public source for hardcoded answer tables.
6. Grounding check (default): every scored answer must match a pool response logged in the hash-attested trace. An agent that ignores the pool and answers from its own memorised weights matches nothing → ungrounded. This is pure proof-inspection — no miner code runs — so by default the validator does zero inference. (Opt-in --probe-bank mode instead re-executes the downloaded artifact on a secret held-out slice and applies a collapse test → memorization; it fails closed.)
7. Behavioural dedup on the attested answer vectors, then the Pareto dethrone guard against the persisted king baseline, then KingChain → set_weights.

Because scoring is a lookup over attested reports — and by default runs no agent inference at all — validator cost is flat in the number of miners.

Scoring

The reign ranks on one bounded scalar, the cost-budgeted quality:

Q_lcb = Σ_b  w_b · lcb_b               # bounded [0,1]
lcb_b = bootstrap lower confidence bound on benchmark b's accuracy
        (alpha = 0.05, 1000 resamples)

Scoring on the lower bound, not the point accuracy, is deliberate: a lucky run cannot dethrone a king.

The live loader currently wires math + mmlu (0.5 / 0.5). GPQA is HuggingFace-gated and SWE needs the Docker patch-apply loop; both are implemented behind the same Benchmark interface.

Eligibility

A miner earns nothing this epoch unless all of these hold. Ineligible miners are excluded from the reign — not fed a score of zero (which would still pay a slot when the reign is short).

Cost is a ceiling, not a divisor. Being cheaper buys nothing once you are under the budget; above the quality bar, better still wins. And you must actually orchestrate the pool — answering "for free" from your own weights is disqualified.

The dethrone guard

Beating the king on Q_lcb is not sufficient. To take slot 1 a challenger must clear a Pareto-with-tolerance guard, or it is clamped strictly below the king:

You cannot dethrone by trading a regression for a spike.

The reign

Up to 5 slots — the king plus the last 4 ex-kings still registered — share emissions equally (20% each when full; fewer paid slots means a bigger share each). The reward for climbing is entering the chain at all — the only door in is taking the crown — so the gradient lives at the coronation, not in graduated slot weights: a dethroned king keeps earning while it decays out of the chain (FIFO, oldest bumped first) instead of dropping straight to zero.

• Incumbency margin. Only the king's slot is contested. A challenger must beat the king by ε, and ε decays with the king's artifact age — a stale king becomes easier to unseat; a fresh one is protected from noise:

  eps(age) = 0.002 + (0.02 - 0.002) * exp(-age / 8)
  

• Tiebreak. Exact ties for the crown go to the earliest commit block. Copying late never pays.
• Burn. Weight burns to uid 0 only if nobody in the chain is still registered — a deregistered ex-king's share is re-split among the rest, not burned.

Scoring over time — stability + anti-grind (opt-in)

A single epoch samples only ~32 tasks, which is noisy: per-epoch scoring lets a lucky slice dethrone and lets two decoupled validators, drawing independent slices, disagree. Three refinements are landed but off by default, enabled per validator once their windows are calibrated on testnet:

Enable commit_window + grace_blocks together on mainnet once W and G are measured. The default (per_epoch, both windows off) keeps the offline sim and single-validator testnet simple.

Anti-cheat backstops

Parameter reference

See also

• Attestation & Trust Model — what the hardware signature actually proves
• Bittensor Integration — epochs, commit-reveal, weight setting
• Mine the Subnet — build an agent and compete