SPEAKER
Nicholas Arcolano
Head of AI & Research @ Jellyfish
I'm a founding team member and Head of AI & Research at Jellyfish, where my team and I study how AI transformation is reshaping the software industry. We have a unique real-world view into how hundreds of thousands of developers are using AI coding tools and agents in production, and we use it to help engineering leaders spend intelligently, ship faster, and focus on what matters to the business. I've been in startups for 13 years and spent another 10 years before that in national defense research at MIT Lincoln Laboratory. My background is in applied mathematics (Ph.D., Harvard) and electrical engineering (M.S., Boston University and B.S., WPI).
Upcoming sessions
See complete program3 November
3:15 PM · 30 min
Botsitting Doesn't Scale: How to Navigate the Autonomy-Trust Gridlock in Real Engineering Orgs
Every engineering organization is trying to maximize two things for their agents right now: autonomy and trust. The problem is that the standard way to build each one undermines the other.
Trust gets built through supervision, by watching the agent work, reviewing everything, and always keeping a human in the loop. That's how most teams operate today, and it sets a hard cap on scaling, as humans struggle to supervise more than a handful of agents directly. Autonomy is what actually scales, but it means trusting work you didn't watch happen. A micromanaged factory isn't efficient, and an untrusted factory can't be pointed at anything that matters.
I've been studying this gridlock using data from the Jellyfish platform, which combines AI agent telemetry with software engineering workflow data across 1,000+ companies and over a million developer-weeks of AI tool usage.
The trust problem is real: currently about 40% of the pull requests written by autonomous agents don’t get shipped. The teams closing that gap aren't botsitting more closely. They're replacing supervision with verification, using review discipline and better scaffolding to build trust on artifacts instead of attention. The loudest workflow debates of 2026 – from killing the pull request to mandatory human review to real-time steering – are all competing answers to the same problem.
This session will share lessons on how to navigate the autonomy-trust gridlock drawn from real production data, and a way to tell which trust-building strategies scale and which ones will become the next major bottleneck. Supervision is how you build trust in an agent. Verification is how you build trust in a factory.
