Skip to main content
← Blog

A 0.6B model went from 10% to 92% on a job you're paying a frontier API to do

· Mick Brzeziński

  • small models
  • fine-tuning
  • AI agents

A developer wanted his home chatbot to route a question to the right shelf — pool, car, hvac, cooking — before searching. He pointed a tiny local model at it. Prompt-only, it got 13 of 131 right. About 10%.

Then he fine-tuned it. The same 0.6B model — small enough to run on a mini-PC — went to 79%, and then to ~92% after one odd tweak. No frontier API. No per-call bill. One afternoon. The writeup is here; the model is Qwen3 0.6B, trained with Unsloth + QLoRA.

The tweak is the part worth stealing. He stopped asking the model for category names and mapped each to a two-character opaque ID with no overlap — KK for hvac, QQ for water heater. Semantic labels were bleeding into each other; meaningless codes couldn’t. A small model has less room to get confused, so you stop giving it ways to.

A bar chart of one 0.6B model (Qwen3 0.6B) categorizing a question, scored on a 131-test battery. Prompt only: 10% correct (13 of 131). After fine-tuning with Unsloth and QLoRA: 79%. After also mapping categories to opaque two-character IDs with no semantic overlap: 92%. The model never changed size — the same tiny local model throughout. What moved the accuracy was training and a labelling trick, not a bigger model or a frontier API.

“Sure, but that’s just classification”

That’s the usual dismissal — tiny models do toy jobs, real reasoning needs scale. The week’s other release kills that take.

Weibo (yes, the social network) shipped VibeThinker-3B — 3 billion parameters, MIT-licensed, free on Hugging Face. On AIME 2026 it scores 94.3; on LiveCodeBench v6, 80.2; on recent LeetCode contests it lands 123 of 128. It matches or beats models hundreds of times its size — DeepSeek V3.2 (671B), Kimi K2.5 (1T) — on verifiable reasoning. Its 1.5B predecessor was post-trained for $7,800, against an estimated $294K for DeepSeek R1.

This isn’t a one-off either. LoRA Land fine-tuned ten small models on real tasks; six beat GPT-4 on their job, all ten beat GPT-3.5.

The catch — and it’s the whole point

VibeThinker has a hard limit, and its own authors lead with it: don’t use it for tool-calling, function calls, or autonomous agents. Ask it broad world-knowledge questions and it trails the giants (GPQA-Diamond 70.2). It is not a small general model. It is a specialist.

The researchers have a name for why: the Compression-Coverage Hypothesis. Step-by-step reasoning leans on a few reusable patterns, so it packs into a tiny core. Broad knowledge — every fact, every long-tail case — still needs scale. Reasoning compresses. Knowledge doesn’t.

So what — if you’re building on this

Look at what your AI feature actually does all day. Most of it is clerk work: route this request, classify this ticket, extract these fields, validate this output, pick the next step. Narrow, repetitive, and checkable — exactly the work that compresses into a model you can own.

The frontier API is a genius you rent by the call. You need it for the open-ended stuff — broad knowledge, fuzzy judgment, agentic reach across tools. But renting a genius to sort your mail is how AI features quietly turn into a per-token tax that scales with your success, on an endpoint a vendor — or a government — can change under you.

The move isn’t “switch to small models.” It’s to split the workload: a $0 specialist you host for the clerk work, the rented genius reserved for the calls that genuinely need it. Drawing that line is a day of work and it changes your cost curve permanently.

So: how much of your AI bill is paying a genius to do a clerk’s job — and which calls could a model you own do instead?