AI speed without the tech debt: the playbook, ranked

Same models, same week, two codebases: one compounds, the other rots. The Carnegie Mellon data says the difference isn’t the model — it’s the harness built around it.

AI doesn’t create technical debt. Ungated AI does. The fix is a feedback loop and a few hard gates around the model.

So here’s the harness I’d build, ranked — because the order is the point.

Same model on both sides — only the loop around it differs.

Do this first: a quality budget your build enforces

If you do one thing, do this. The CMU failure wasn’t that AI wrote worse code — it was that nobody saw the rising warnings and complexity until velocity had already died in month three. The debt accumulated in the dark.

A budget makes that impossible. Put a quality gate on every pull request — complexity, duplication, and static-analysis warnings on the changed lines — and fail the build when it regresses. A ratchet: the number can only hold or improve, never climb. No green check, no merge.

This neutralizes the exact two metrics the study found persistent, costs an afternoon to wire up (SonarQube, Qlty, or your linter with thresholds), and needs no behaviour change from anyone. Everything below makes the green check easier to earn — but this is the check.

High impact: move correction into the loop

Put lint, type-check, and tests inside the agent’s loop — not after it. This is the harness ThoughtWorks named: the agent runs the checks, reads the failures, and fixes them before a human ever sees the diff. Volume stops turning into debt, because every chunk is corrected while context is still loaded. Give the agent the commands and a project guide (a CLAUDE.md, a Makefile) and it closes its own loop.

Make tests the merge gate, and pin behaviour before you let AI refactor. A characterization (golden) test locks in what the code does now; then the agent is free to simplify, because the test screams if behaviour drifts. The model stays on gap-closing work, under a check it can’t talk its way around, instead of owning the whole system — the ProgramBench line.

Solid: keep a human able to say no

Ship small, atomic diffs. Big generations are where complexity hides and where review degrades into rubber-stamping. One concern per PR, capped in size. A diff a human can hold in their head is a diff a human can actually reject.

Review every line — treat AI output like a junior’s PR. The accept-without-reading reflex is the human-side version of the invisible debt: it merges fine and rots later.

A human owns the architecture; the AI fills it in. ProgramBench is the evidence: models drift toward monolithic, single-file implementations, so the system-level calls — module boundaries, contracts, where state lives — stay with a person. The agent builds inside the lines you drew.

Compounding: institutionalize the cleanup

Budget pay-down time. Month one generates debt by design — so schedule month two to pay it, before the velocity curve sags. A recurring “debt sweep” — a sub-agent whose only job is to dedupe and break up the functions that tripped the complexity budget — turns cleanup from a someday into a cadence.

Orchestrate roles, and spend the expensive model narrowly. A planner scopes, an executor writes, a reviewer agent reads the diff for complexity and duplication before a human does. Point a cheap model at the wide first pass and the premium one only at what survives — cost is the shape of the pipeline, not a setting.

Most teams reach for the bottom tiers and skip the top one. Reverse it.

The trap: a tool is not a harness

Buying Cursor or Copilot and skipping the loop isn’t a smaller version of this playbook — it’s the CMU experiment, re-run on your own codebase. The tool reliably delivers month one’s +281%. Only the harness decides whether month three keeps it or eats it.

So what

The edge moved. “Does your team use AI” is table stakes now — and on its own it buys one good month and a quiet pile of debt. The real edge is the harness around the model: not a tool you purchase, but a discipline you build.

If you fired the model tomorrow, would the codebase it left behind still be one you’d want to build on?