Resources // TERSE

Benchmarks

Numbers with their methodology attached. Two studies so far: encoding efficiency (how many tokens TERSE saves over JSON on the same state) and a full agentic head-to-head — terse-brain against Karpathy's llm-wiki, out of the box, on identical work. Measure your own data in the playground.

~2.5×
Token efficiency vs JSON
6.5×
Fewer tool calls (brain bench)
3.5×
Faster wall time (brain bench)
2.9×
Lower cost per run (brain bench)

S1Encoding efficiency — TERSE vs JSON

The claim: representing the same state in TERSE instead of pretty-printed JSON costs roughly 2.5× fewer tokens under GPT-class tokenizers. The mechanism is structural — JSON taxes every key with quotes, braces, and repetition; TERSE puts identity in names, drops the punctuation tax, and collapses booleans to flags.

Payload shapeTypical savingsWhy
Nested objects with prose fields2.5–3×TEXT blocks carry prose without escaping; nesting collapses to # depth
Entity records with booleans & enums2–2.5×Booleans become flags; short enums stay unquoted
Flat uniform tables1.5–2×Least advantage — tabular data has little key overhead to remove
Honest scope: savings depend on your data's shape and your model's tokenizer. That's why the playground exists — paste your own payloads and read your own number. The figures above reflect our internal corpus of mixed agent-state payloads.

S2Agentic head-to-head — terse-brain vs Karpathy llm-wiki

The stronger claim isn't token counts — it's what happens when an agent has to work the state. We ran the same knowledge-base workload through terse-brain and through a faithful out-of-the-box Karpathy llm-wiki (markdown) setup, and measured everything.

Setup

ParameterValue
ArmsT = terse-brain · K = Karpathy llm-wiki (markdown), out of the box
Workload15 documents ingested · 24-question schedule · 3 reps = 72 graded answers per arm
ModelSame model for both arms; graded by an independent LLM judge (Grok 4.5)
Question tiersT1 direct recall → T4 abstention & contradiction handling

Accuracy — parity

ArmPassRate
T — terse-brain66 / 7291.7%
K — Karpathy wiki67 / 7293.1%
One question of difference at n = 72 — statistical parity. Both arms shared the same weak spots (one authored-docs tier question and one abstention case where a regex check failed an answer the judge liked). The point is not that TERSE answers better; it's that it answers as well for a fraction of the machinery below.

Efficiency — where TERSE wins

Metric (mean per rep)terse-brainKarpathy wikiAdvantage
Input tokens~1.81M~5.17M2.9×
Output tokens~59k~409k6.9×
Tool calls~109~7096.5×
Wall time~23 min~80 min3.5×
Est. API cost / rep~$2.10~$6.092.9×
Est. cost, 3-rep total~$6.31~$18.272.9×

Why the gap: markdown can't enumerate or query itself, so the K arm greps, re-reads, and rewrites whole files — 709 tool calls of it. The T arm asks ? [DEPTH 2; CONTAINERS] for its index and patches by declaration, so most of the bookkeeping simply never happens.

Caveats — read before quoting

#Caveat
1n = 72 questions per arm; single workload at personal-knowledge-base scale (~15 sources). Not a claim about terabyte corpora.
2One model family per run; an LLM judge (Grok 4.5) graded answers. Judge choice can shift absolute pass rates.
3Costs estimated at $2/M fresh input, $0.30/M cached, $6/M output. Your rates will differ.
4TERSE 0.2.8 is pre-1.0; the harness and results (report.md / report.json) live in the repo so you can rerun as the language evolves.
Reproduce it: the benchmark harness ships in the terse-lang/terse repo — same docs, same question schedule, your model and judge of choice.