Why Spec Driven Development Didn't Kill Code Review at HumanLayer

Dexter Horthy spent a chunk of last year telling anyone who would listen that reviewing AI-written code was becoming optional, so long as you had a good enough plan. On a recent episode of The AI Native Dev, the HumanLayer CEO and co-founder walked through why that bet on spec driven development broke his own codebase, and why he's landed somewhere more nuanced: specs matter enormously, but they aren't yet a replacement for looking at what the model actually built.

The story starts in the early days of the Claude Agent SDK, when HumanLayer's team noticed something that Horthy now calls the "planning metagame." A written plan, it turned out, was the single most effective way to get a model to work for longer without going off the rails. Part of that was pattern-matching to how the models behaved back then. Horthy coined the term "the dumb zone" for the point, around 100,000 tokens at the time, where a model's output quality would fall off a cliff. Write a good plan, keep the context tight, and the agent stayed useful for longer stretches. Push too much into its context window, and you got worse results, not better ones. It can be helpful to think of that dumb zone as still present today, just further out: too much focus, Horthy argues, is its own kind of no focus. That's the core problem context engineering exists to solve, deciding deliberately what belongs in a model's context and what doesn't, rather than leaving it to chance.

That planning instinct collided with an idea making the rounds in AI engineering circles: Sean Grove's framing, from his time at OpenAI, that the spec is the new code. In Grove's analogy, a spec would eventually be to code what source is to a compiled binary, the durable, human-authored artifact, with the code itself becoming closer to an intermediate output. HumanLayer took that idea further than most. For a stretch, Horthy's team ran an internal experiment where, in his words, "if you read the plan, the code is not as important." Engineers reviewed the plan in detail, then let the agent implement it largely unsupervised.

The problems showed up slowly, then all at once. HumanLayer ran the approach for four or five months before realizing the codebase had become unusable. Every change caused a regression somewhere else. Part of the failure, Horthy suggests, came from also letting models make architectural decisions with limited human oversight, an experiment that produced what he now describes as an "incredibly complex system": a desktop app talking to a Golang daemon over a Unix socket, which launched and managed coding-agent sessions, which in turn spun up MCP servers to handle approvals, writing to a SQLite database that fed back through the same chain to the UI. Functional, but nobody had chosen to build it that way on purpose.

Eventually the team rebuilt the product from scratch, and for the first two weeks of that rewrite, Horthy's co-founder Kyle Mistele sat in an editor typing out the new data architecture by hand, character by character, no agent involved. The instinct behind it maps to a line Horthy attributes to an old rule from early C programming: to debug code, you have to be smarter than you were when you wrote it. Write the cleverest possible code and, by definition, you're not smart enough to fix it when it breaks. The system you build has to stay simpler than your own peak understanding of it, which is a harder discipline than it sounds once an agent is doing the typing.

None of this means Horthy has reversed his position on specs. He still describes them as a slider between determinism and adaptability: how much of the outcome do you want to dictate, and how much room do you want to leave the agent to make its own calls within that frame. The way to actually get good output isn't to review the code more carefully, in his view. It's to get better at capturing intent up front, and at defining verifiers that let the agent check its own work, what he calls "back pressure." A model with a fast, reliable way to tell if it's on the right track will, in his words, do dramatically better work. The planning-versus-yolo tradeoff, in that framing, comes down to expected pain: a two-sentence prompt is cheap but carries real odds of rework, a five-hour handwritten spec costs more up front but leaves less to chance, and the right point on that curve depends on how much it would hurt to be wrong.

Where Horthy is more skeptical is of the idea that code review disappears entirely, at least soon. He points to a benchmark, run by a university lab, that tests whether a model can keep extending a real codebase across many sequential features without the whole thing degrading. The leading results are still under 20%. Models have gotten better at plenty of things this year, Horthy notes, computer use, video generation, but maintainable long-horizon coding hasn't moved nearly as fast as the discourse suggests. Part of the reason, he argues, is that there's no fast oracle for maintainability: you often don't find out a codebase has become unmanageable until months later, which makes it a genuinely hard thing to train a model against.

HumanLayer's current answer looks less like pure spec-driven trust and more like a small software factory. The team runs a handful of scheduled agents overnight, deterministic checks alongside softer pattern-matching ones, that open pull requests fixing small things before anyone reviews a line of new code. The philosophy, as Horthy put it, is to keep shifting attention from your position, the code in front of you right now, to your velocity, the system that keeps producing good code over time.

He's not ready to say review goes away completely, either. A story he shared from a recent dinner in San Francisco stuck with him: someone who'd been building agent systems for a long time argued there will probably always be alpha in reviewing something, even once code itself stops being the primary artifact developers touch day to day. Nobody knows yet what that something will be. But teams that skip review entirely, in Horthy's view, tend to converge on whatever the model would have produced for anyone else asking the same question, and there's little differentiation in that.

Worth a listen for anyone trying to figure out where their own team should sit on that determinism-to-adaptability slider, and for anyone who has already lived through a version of HumanLayer's rebuild.