Don't trust the AI leaderboard they just pasted into the pitch

Meta tested 27 private variants of Llama-4 on Chatbot Arena before release, then published only the one that scored best. The chat-tuned “experimental” version it floated shot up the rankings. The model you can actually download finished 32nd, below GPT-4o and Claude 3.5 Sonnet.

That’s not a one-off scandal. It’s what happens when a measure becomes a target — and right now the public AI leaderboard is measuring almost everything except capability.

The board is tilted before anyone votes

“The Leaderboard Illusion”, a 2025 study, took apart how Chatbot Arena actually works.

First, private testing pays. Submit a batch of variants, keep the best, bury the rest, report only the winner. You’re not building a better model; you’re sampling for a luckier one.

Second, the data isn’t shared evenly. Google and OpenAI each pulled an estimated ~20% of all Arena battles; 83 open-weight models combined got 29.7%. And that data pays: the paper estimates even limited extra Arena data can buy up to a 112% relative gain on its distribution. The biggest labs ride that feedback loop; a small team can’t.

The votes reward length, not correctness

Even an honest vote has a bias baked in. When LMSYS controlled for style — answer length, markdown, bold, lists — the ranking moved. Length was the biggest style factor (regression coefficient 0.249); GPT-4o-mini dropped from 6th to 11th once you controlled for style.

So the lever is obvious: make answers longer and prettier and you climb, capability unchanged. That’s exactly what a leaderboard-tuned “experimental” submission is built to do.

And the test answers are in the training set

Then there’s contamination. Scale AI built GSM1k — a fresh set of grade-school math problems matched item-for-item to the well-known GSM8k — and re-ran the models. Accuracy fell by up to 8%, with several model families showing “systematic overfitting.” When a benchmark has been on the open web for years, a high score can mean the model memorised the answer key, not that it can reason.

This is why the people who build evals for a living now design for it. LiveCodeBench only scores models on coding problems published after their training cutoff; LiveBench swaps in new questions every month.

And many of those classics are simply finished. GPT-4 hit 86% on MMLU in early 2023, and every frontier model since lands within a few points of it. A benchmark everyone aces tells you nothing about who’s ahead.

So what — if you’re choosing a model or a vendor

The launch-day chart is marketing, and increasingly it’s marketing about a model you can’t buy. By the time a number reaches your slide deck, most of what made it move has nothing to do with how the model will perform on your work.

The fix is the same one that makes any AI build defensible: measure on your own task. Take ten real jobs from your actual workflow, write down what “good” looks like, and run the candidates through your own harness. That private eval is boring, it never trends on X — and it’s the only score that predicts your result.

So before the next leaderboard screenshot talks you into a migration: do you have ten of your own tasks scored, or are you still trusting someone else’s?