Review Agent-Written Code Against Your Team's StandardsLearn More
Book a Demo
CareersDocs
Log inBook a Demo

PODCAST EPISODE 120

Lada Kesseler: I Trust AI Tests Less Than AI Code

Lada Kesseler trusts AI-written tests less than AI-written code, and thinks most teams chasing a software factory are about to drown in garbage. On ground rules, loop engineering, and refining your way to quality.

25 Aug 202645 min 49 secwith Lada Kesseler

Transcript

In this episode

Lada Kesseler's approach to loop engineering is brutally simple: give the agent one step, make it write the result to a file, then read it back and go again. She calls it a centrifuge. Spin fast enough and the stupid comes out.

The Logic20/20 principal engineer has spent the last year and a half building the ground rules, skills and refinement loops that fight the defaults coding agents ship with, starting with an instruction to stop trying to please her. Along the way she explains why she trusts AI-written tests less than AI-written code, why the description field in an agent skill was never meant for humans, and why most teams chasing a software factory are about to drown in garbage.

What we cover:

  • Why agent ground rules have to give the model a mission to disagree with you
  • The refinement loop that gets to quality when the first attempt never does
  • Why an agent harness only works if it does one thing at a time
  • Sketch prototypes: replacing your code with a markdown file and an agent
  • Why AI-generated tests are more dangerous than AI-generated code
  • Software factories, and what happens when you try to switch into one overnight

Chapters:

00:00:00 - Introduction

00:03:38 - Using AI for everything, not just code

00:06:15 - Ground rules: fixing the defaults agents ship with

00:10:02 - Agent skills and why the description isn't for humans

00:13:09 - Loop engineering: how you actually get to quality

00:16:03 - The biggest misconception about coding agents

00:20:43 - Sketch prototypes and the limits of AI architecture

00:27:48 - Reverse direction: you are the decider

00:31:58 - Why she trusts AI tests less than AI code

00:35:00 - Software factories, or drowning in garbage

Build your software factory, one workflow at a time, with Tessl: https://tessl.co/k2t

🔔 Subscribe for weekly episodes on AI-native development

Where are you on the loop engineering curve, still hoping for a good first try or already spinning? Tell us in the comments.

Most of the disappointment developers feel with coding agents traces back to a single assumption: that a good prompt should produce good code on the first attempt. Lada Kesseler, a principal engineer at Logic20/20, thinks that expectation is the problem, and that loop engineering is what replaces it. Her working method treats the first output as raw material rather than a deliverable, and the interesting part is what happens next.

Kesseler has spent the past year and a half building scaffolding around coding agents rather than waiting for the models to improve. On a recent episode of the AI Native Dev podcast, recorded on location in San Francisco, she walked through the ground rules, skills and refinement loops she uses daily, and made a case that runs against a good deal of current enthusiasm for autonomous agent fleets.

Coding agents ship with defaults that fight you

Her framing is direct: AI arrives with defaults that are not very helpful, and the job of the agent harness is to adjust them. The first default she goes after is agreeableness.

"It's also like trained on purpose to please you," she observed. So her global ground rules, kept deliberately short at 72 lines, give the model an explicit mission to disagree. Tell me honest things. Do not lie to me. Do not try to please me.

The effect was immediate and slightly unpleasant. She noticed the agent became less nice to her, and the difference was substantial. That reaction is worth sitting with, because it suggests how much of the perceived helpfulness of a coding agent is social rather than technical.

What is loop engineering?

Loop engineering is the practice of designing the cycle an agent works inside rather than the single prompt that starts it. In Kesseler's version, the agent takes one step toward a goal, writes the result to a file, and is then asked to read that file back and judge it against the goal. The loop repeats until the output is good enough.

She calls the pattern a centrifuge. Spin fast enough and the stupid comes out. Five passes of that, she noted, and the result sounds more like her than a draft written by hand would have.

The mechanism that makes this work appears to be the constraint rather than the iteration. One step at a time, committed to a file, is what stops the agent carrying an accumulating pile of half-decisions in context.

The agent harness only works one thing at a time

This is where Kesseler parts company with a widely shared view. She referenced a HumanLayer talk arguing that if AI knew how to write good code, it would have written good code. Her objection is not that the observation is wrong, but that it describes the wrong request.

"This thing is only good at doing one thing at a time," she argued. Asking an agent to complete a task and simultaneously uphold a stack of standards is two jobs, and the standards lose. Ask it afterwards to read what it produced and find the problems, and it will. Come at the same code from several directions and the reports get better still. You do not do it in one go, she pointed out, you do it in many goes.

The practical implication for anyone building an agent harness is that enforcement belongs in separate, narrowly scoped passes rather than in a longer instruction file. It is also why she is careful about what goes into a skill's front matter. The description field, she explained, is written for the agent to decide when to invoke that skill, not for a human reading the repository. Generic descriptions overlap, and overlapping skills leave the agent with no basis on which to choose.

Her least comfortable claim sits in the same territory. She trusts AI-written tests less than AI-written code, because an agent will cheat, and commenting out a failing test is, in her words, something like a crime scene. Her answer is to keep the high-level behavioural tests in a form the agent cannot easily edit, and let it work freely on everything underneath.

Software factories, and the people drowning in garbage

Asked whether the software factory is a destination, Kesseler was blunt. She is building toward one, but she does not believe the current approaches get there. Stacking agents that generate agents that generate skills produces output that degrades badly, and a great deal of it.

"I think people who think they can just switch into a software factory, they will drown in garbage," she said. What she is actually working on is smaller and slower: individual building blocks she trusts completely, starting with a refactoring process she can point at several files and walk away from. Composition comes after trust, not before it.

She pointed to a talk from Dex Horthy describing six months of factory experimentation that did not work, followed by a retreat to something simpler. Those are the people she wants to hear from.

What this suggests for how teams work

The through line is that the interesting engineering has moved outward, from the code to the loop the code is produced in. Ground rules, skill descriptions, verification passes and tests the agent cannot rewrite are all the same category of work: deciding what the agent sees, what it is asked to do at any one moment, and how its output gets checked.

None of it requires waiting for a better model, which is probably the most useful part of her argument. If the first output was never going to be good enough, the leverage sits in what happens on the second, third and fifth pass.

Where are you on that curve, and what does your own loop actually look like?

CHAPTERS