Merge Rate Measures Whether AI Adoption Is Real

The horror-story version of my talk title was "When our PM started writing code." But the real question was not whether product managers and designers can open pull requests. The real question was whether organizations know if those pull requests are meaningful.

Every company wants to become AI-native. When I ask what that means, the answer is often vague. Some people say it means PMs, designers, QA, and other non-engineers can open PRs. That is partly true, but it is a symptom rather than the definition.

My definition is this: an AI-native organization collapses the handoff. The person who cares about the outcome, and has the authority to decide, can also do the work through agents.

My talk, "When Our PM Started Writing Code: What Merge Rate Taught Us About AI Adoption," was about how to make that shift measurable. PR count is not enough. Token spend is not enough. Merge rate starts to tell you whether the new workflow is actually working.

Use This Talk As Agent Context

Tessl has turned my AI Native DevCon talk into a skill your agent can use as context. You can also watch the full recording.

The Handoff Is The Bottleneck

For many organizations, the bottleneck is no longer how fast a developer can type code. It is the handoff from idea to design to ticket to sprint to implementation to product review.

A product manager has an idea. A designer turns it into a Figma. A developer picks it up a sprint or two later. Then the team discovers the ticket missed something, the design needed another pass, or the implementation does not match what the PM had in mind. The iteration loop is long.

In an AI-native workflow, the person who cares can execute much earlier. A PM can build and tune a user-facing change with an agent. A designer can make the pixel-level iteration directly. An engineer can focus more on architecture, dependencies, and system shape.

That does not mean everyone does everything. It means the first handoff shrinks from days or weeks to minutes. By the time a developer sees the work, product review and UX iteration may already have happened.

Spending More Is Not Adoption

The wrong way to become AI-native is to spend more on the same people doing the same tasks.

I mentioned companies where AI budgets grew quickly but the business could not see a clear link to faster feature shipment. If developers get higher token limits but are still assigned work they do not care about, they may hand it to an agent and let the agent loop. The bill goes up, but the organization has not changed.

Giving PMs and designers generic developer tools can fail too. Claude Code is a good tool for developers, but "code" is in the name for a reason. Non-technical people may need help setting up environments, may produce huge PRs, or may increase PR fatigue for developers if the workflow is not designed for them.

Prototyping tools alone can also miss the bottleneck. If a PM can create a prototype faster but still has to wait for the same handoff into engineering, they have simply created the signal earlier and then waited longer.

The right pattern is to give the people who care tools tuned to their role and connect those tools to the real codebase through a harness that protects quality.

The Right Boundary Is Authority

Once agents let anyone change almost anything, 'who can' stops mattering and 'who should' takes over. The boundary is authority: do the work where you own the decision, not where you don't.

A PM or designer should be able to make user-facing changes where they own the judgement: copy, layout, flows, visual refinements, small UX changes, and product details. Engineers should still own engineering decisions, architecture, data models, back-end design, security posture, and the parts of the system where their judgement matters.

The boundary is not always obvious. I gave an example where a designer opened a PR for a feature that displayed images for different versions of a page. In product review, it worked exactly as intended. In engineering review, it failed because the agent stored the images server-side in a way that would lose them the next day. The developer rightly said the team needed to decide whether storage belonged in a database, a bucket, or somewhere else.

That PR did not mean the workflow failed. It accelerated the conversation. The user-facing work could still be reused, and the engineering decision happened with a concrete example in front of the team.

The expected merge rate should not be 100%. Some overstep is healthy because it shows where the boundary is.

Harness Engineering Creates The Confidence

The confidence to let non-technical contributors touch a brownfield codebase comes from the harness.

A harness is the system that makes repeated agent mistakes harder to repeat. The agent makes a mistake. The mistake is noticed. The harness changes so the next agent gets feedback earlier.

For non-technical contributors, the harness has to do several jobs. It needs to onboard itself into the codebase. It needs to understand product-level language and map that to code. It needs to communicate with the user without requiring them to point to files and line numbers. It needs to manage long sessions, check its own work, know what good means, and prove what it did.

Most importantly, the harness should own the code constraints so the PM does not have to. Product people should not need to memorize component boundaries, coding standards, and deployment quirks every time they ask for a user-facing change.

The agent should learn from all users in parallel, not only from one session. That is how the system improves with usage.

Measure PR Count First

The first metric is how many pull requests are opened by non-technical contributors, and by which people.

This tells you whether the organization is actually enabling people beyond engineering. Some PMs and designers will be confident early. Others will be hesitant. If only one person is using the system, the organization has not changed.

But PR count alone is dangerous. It is easy to increase PR count by generating low-quality work. That creates PR fatigue and pushes hidden cost onto developers.

So PR count is only the top of the funnel.

Merge Rate Is The Quality Signal

The second metric is merge rate.

If non-technical contributors open PRs and most are rejected, the workflow is not healthy. Either the tool is not respecting the codebase, the contributors are overstepping authority, or the review path is not clear enough.

Across the organizations and PRs we analyze, an average non-technical contributor working through our system has a merge rate of about 74%. That means roughly one in four PRs oversteps or is not ready. I see that as acceptable. It is enough to preserve trust without pretending the boundary is perfect.

The important thing is to avoid PR fatigue. Developers should feel that most PRs from the workflow are worth reviewing.

Track Developer Follow-Up Work

The third metric is how often merged PRs need extra developer commits before they can land.

A PR may eventually merge, but if a developer had to rewrite it, fix the architecture, or patch the agent's output, the cost did not disappear. It moved into review and cleanup.

For us, about 84% of merged PRs from non-technical contributors merge without a developer adding extra commits. That is a useful signal that the workflow is not adding too much hidden burden.

This is where traceability matters. Our system can track the authoring user behind the bot identity and inspect the commit stream before merge. That makes it possible to distinguish a clean agent-authored PR from one that needed engineering intervention.

Reduce PR Fatigue With Risk And Size Labels

A quick shift: the rest of these are practices, not product. Patterns we and the teams we work with landed on in the field for keeping PR load sane once more personas are shipping.

One practical pattern is to label every PR with size and risk. If a reviewer sees three small low-risk PRs and one extra-large high-risk PR, they can allocate attention sensibly. Without those labels, every PR feels like another interruption.

This is also where smaller PRs matter. The agent should reuse existing components, model the codebase, and make only the necessary changes. A non-technical workflow that produces giant PRs will lose trust quickly.

Feature flags are another practical part of the workflow. Developers may move faster when they can make product decisions, merge behind flags, and let product or UX follow up with refinements in later PRs. Fast flow often means accepting that a feature may become three or four PRs rather than one perfect handoff.

AI-Native Adoption Is Democratized Authorship

The moral of the talk is that AI-native adoption is not bigger token budgets. It is democratized authorship.

The accountable person should be closer to the work. PMs and designers should be able to execute where they have authority. Engineers should spend more of their time where their judgement matters most. The harness should keep the organization inside its quality boundaries.

Merge rate is useful because it exposes whether that system is actually working. It connects adoption to review quality, trust, and hidden developer cost.

If you only measure spend, you will see the bill. If you only measure PRs, you will see motion. If you measure merge rate and developer follow-up, you start to see whether AI adoption is becoming real organizational capability.

The full version of this argument was presented at AI Native DevCon London. To go deeper, watch the full recording.