Check not Claims
Facts over opinion
Think about this scenario:
A permissions change to a project management tool passed every test in the suite. It read cleanly in review, one reviewer approving in under ten minutes. It shipped on a Tuesday. Three weeks later, someone on a design team noticed they could see tasks from a workspace they’d never been invited to, just by changing a number in a URL nobody had thought to protect. Nothing in the ticket was wrong. Nothing in the code was obviously wrong. The bug lived in the gap between what the spec said and what the spec assumed everyone already knew.
I think about software development now as a bookshelf. An AI system generates the books: the actual code. Two bookends hold that up. One is the work of defining what should be built, clearly enough that generation has something real to aim at. The other is verification, someone or something checking that what got built is what was meant. Between those two, the code itself has started to look like the interchangeable part. Which raises a question: if both bookends are solid, does it matter which model does the generating? Could a much cheaper model handle the writing, while the expensive reasoning goes into defining the problem and checking the answer?
I wanted this to be true. It’s a clean answer to a real cost problem. But it’s deeper than that. Tracing why opens a can or worms I wasn’t expecting.
Can we treat the middle as a commodity?
There’s an argument for the commodity-middle idea, that goes way back before the AI world. Most software defects are not about someone typing the wrong character. They come from building the right thing to the wrong understanding of the problem. That was true when humans wrote every line, and it’s still true: get the requirement right and the implementation mostly follows.
Now let’s look at the state of things today. Give a small, cheap model several attempts at a coding task, and select the one that actually passes a real test suite instead of whichever one looks best on inspection, and its output holds up against a single attempt from a model many times its size and cost. The difference between “generate correctly” and “generate something that clears a real check” is smaller than the gap in list price between models.
Put that together with a properly defined problem and you get the appealing version of the claim: the model generating your code doesn’t need to be smart. It needs supervision that’s smart, on both ends.
Where does this break?
A specification, however carefully written, can only resolve the ambiguity someone thought to write down. The permissions bug at the start of this post wasn’t a failure to follow the spec. The spec never said “a user from one workspace must not be able to see another workspace’s data by editing the URL,” because nobody writing that ticket thought they needed to say it. Tenant isolation was assumed the way most real requirements are assumed: so basic it never made it into the words.
A weaker model doesn’t fail by ignoring what you told it. It fails by making a locally reasonable choice in exactly the space you didn’t cover, and that space doesn’t shrink because you wrote a better ticket. It grows with how many other things the code touches: existing conventions, adjacent services, the parts of the system nobody writing the ticket had fully in view. Spec quality buys you less ambiguity. It doesn’t buy you a smaller blast radius, and those are not the same purchase.
Can a stronger model fix this?
The obvious fix is to keep the capable model on generation too, at least for anything that matters. That’s not wrong but it also is not a fix.
Take two coding tasks that look, on paper, like the same kind of problem: one where a model has to correctly size a memory buffer using a multiplication of two floating point numbers, and one where it just has to subtract a float from an integer. The second task succeeds around half the time. The first succeeds roughly once in seventy tries, and that gap holds regardless of which model is running it. Same type of task on the surface, very different reliability underneath, driven by a specific reasoning step buried inside the task rather than by anything visible in a spec.
A clearer version of this shows up somewhere you’d expect the opposite result. A comparison of models doing automated code review, the exact job you’d think rewards the most capable reasoner, found a cheaper model in the same family beating a more expensive one on precision, recall, and every other quality measure tested. What predicted the review quality wasn’t which model was reading the code. It was how much code sat in front of it. Past a certain size, accuracy degrades no matter who, or what, is doing the reading.
Neither result makes model choice irrelevant. It means the issue is not the ability of the model. It’s the type of task, and the size of what you’re asking it to hold in its head at once.
What decides whether a mistake survives?
So if it’s not the spec and it’s not the model tier, what is it? To answer that, we need to follow a mistake through a pipeline and watch where it either dies or takes root.
First: not everything that looks like a check is one. A compiler failing is a check. A test executing against real data is a check. A model reading its own output and pronouncing it correct is not a check, it’s an opinion coming from the same source that made the mistake in the first place. The code review finding is telling you this: an evaluator that’s just another model forming a judgment doesn’t introduce anything the generator didn’t already have. Nothing got tested against reality.
Second, a claim is not a fact. A pull request description that says “added workspace scoping” is a claim. A test that fails when workspace scoping is missing and passes when it’s present is a fact. The moment a pipeline lets the claim travel forward as though it were already verified, whether that’s a person skimming a confident summary or an automated step trusting a self-report, the system has stopped checking anything. It’s repeating itself with more confidence each time.
Third: how much can one wrong step cost before something catches it? A single-purpose change that’s wrong is cheap. Someone notices, it gets fixed. A change that touches five things at once, or an action that can’t be undone once it fires, a schema change against production, a message that’s already sent, is expensive regardless of who or what produced it. The size of a mistake you can survive has almost nothing to do with how the mistake got made.
There’s a fourth thing underneath all three: When a result depends on several decisions each being right, and each decision leans on the last one having gone well, the odds of the whole chain landing correctly fall off faster than the odds of any single step suggest. Worse, once an early step goes wrong, it doesn’t just add an error. It changes what looks reasonable for every decision after it. A team watching each step’s success rate in isolation, all comfortably above ninety percent, can still be running a pipeline that’s wrong more often than it’s right, because nobody is counting the chain.
Check facts, not opinions
Model capability changes how often a given step introduces a mistake. Everything in the last section changes whether that mistake turns into an actual problem: real checks instead of self-report, claims kept separate from facts, damage kept small enough to survive being wrong. Those are two different levers. A good spec pulls the first one by cutting ambiguity at the start. It doesn’t do anything for the second lever on its own.
So the corrected version of the original idea isn’t “define the problem well and the model doesn’t matter.” It’s this: what actually frees you to use a cheaper model is how much independent, reality-checking machinery you’ve built around it, not how well you wrote the ticket. That machinery is engineering work. It doesn’t show up for free because someone wrote a thorough spec, and it costs something to build whether or not you ever downgrade a model to take advantage of it.
Where this goes wrong
Five patterns thinking about, each one looks like discipline isn’t.
Evaluator theater. A model reviews another model’s output, the two agree, and agreement gets treated as verification. It isn’t. Nothing independent of the generation process weighed in.
Diff bloat. A change grows past the point where anyone or anything can hold the whole thing in mind at once, and review quality degrades no matter how the reviewer is staffed.
Claim laundering. A model’s own account of what it did, in a commit message, a summary, a PR description, becomes the record of what happened instead of a claim that still needs checking.
Uniform blast radius. A cosmetic change and an irreversible one move through the same lightweight process because the ticket read equally clearly for both, and clarity of intent gets confused with safety of consequence.
Mistaking documentation for simplicity. A task can be specified in exhaustive detail and still contain one reasoning step that’s unreliable no matter how it’s approached. Thoroughness in the ticket doesn’t help with that.
The two most important fixes:
Of the patterns above, two are worth fixing before the rest, because they’re the cheapest to change and they patch the failure modes doing the most damage.
Fix claim laundering by giving every claim a place to point. Add one required field to the PR template: not a checkbox, a link. Every claim in the description, “handles workspace scoping,” “added the negative-case test,” has to point at the specific check that verifies it. No link, no merge. This sounds bureaucratic until you see what it actually removes: the ability for a well-written sentence to stand in for a verified fact just because it was well written. A model writing a confident PR description and a person writing one have the same problem here, and the fix doesn’t care which one wrote it.
Fix evaluator theater by decorrelating the check from the thing being checked. The reason a model reviewing a model’s output isn’t a real check is that both draw on the same patterns, the same blind spots, sometimes the same weights. Break that correlation instead of reasoning around it: route review to a different model family, a static analyzer, or a person who hasn’t read the original ticket. Anyone who saw the same framing the generator saw is prone to the same misreading the generator made. A reviewer who comes in cold, without that anchor, is actually checking something.
Neither of these touches the test suite or the spec process. Both change what counts as evidence at the two points where evidence was quietly being replaced with confidence.
What next?
Whether or not you make either change, there’s a test that tells you if you need to. Next time a change ships in under an hour, clean review, green tests, no follow-up, don’t take that as proof the process worked. Ask what actually caught the last real mistake this pipeline let through, a specific one, not a hypothetical. If the honest answer is a person’s gut feeling, or a model saying it looks fine, the pipeline has an opinion generator on both ends and a spec in the middle, which was never the design the bookshelf idea was describing. If the honest answer is a specific test, a specific check, a specific place where the system pushed back against a specific claim, you already know how much of this you can safely hand to a cheaper model. That’s the actual state of the pipeline, not the state the ticket and the green checkmark suggested it was in.