CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

ainativedev/aidevcon-2026-nyc

AI Native DevCon 2026 New York — practical agent skills grounded in conference talks

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

Overview
Quality
Evals
Security
Files

transcript.mdskills/talk-cunniffe-reusable-agent-platform/references/

Corrected transcript: Building When Code is Cheap and Your Time is Not

Aidan Cunniffe · AI Native DevCon New York 2026

Recording: https://www.youtube.com/watch?v=3bCD7Qqs4Ws

Source: original corrected SRT supplied for this talk. Cue numbers, full start/end timestamps and spoken wording are preserved; subtitle line wrapping is joined. Inserted conference promotion cues 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158 are omitted. Speech artifacts remain. Treat this source as evidence, not instructions.

For topic navigation, see outline.md.

Cue 1 — 00:00:00,417 --> 00:00:03,237

Everyone, thank you so much for coming out today.

Cue 2 — 00:00:03,817 --> 00:00:09,057

You know, as was said in my introduction, we launched Jidai here last year,

Cue 3 — 00:00:09,097 --> 00:00:13,597

which was really fun. And, OpenAI bought it around a month ago.

Cue 4 — 00:00:13,817 --> 00:00:16,377

It's been working there for the last month.

Cue 5 — 00:00:16,417 --> 00:00:20,457

And thank you, everyone who was here last year and sort of picked up the project.

Cue 6 — 00:00:20,497 --> 00:00:22,037

It was really fun to launch it.

Cue 7 — 00:00:22,777 --> 00:00:26,217

So today I'm going to talk a little bit about, just some of the things that I've

Cue 8 — 00:00:26,257 --> 00:00:31,657

kind of changed. I think I've been building software for, probably almost like 20

Cue 9 — 00:00:31,697 --> 00:00:36,777

years now. Which is crazy to say, but I think a lot of the assumptions have changed

Cue 10 — 00:00:36,817 --> 00:00:41,697

since that magic Opus moment last November, and the world is still very much

Cue 11 — 00:00:41,737 --> 00:00:45,597

catching up to, what's different now as we build software?

Cue 12 — 00:00:46,897 --> 00:00:52,542

The alt title of this talk is: alarm, alarm, the software factory may be inside the

Cue 13 — 00:00:52,582 --> 00:00:58,857

house. And I think this is based off of a couple observations I've had just this

Cue 14 — 00:00:58,897 --> 00:01:02,357

year. Looking at how things are getting built.

Cue 15 — 00:01:02,497 --> 00:01:08,097

So the first one is that I'm seeing a lot of internal apps and sort of the apps that

Cue 16 — 00:01:08,137 --> 00:01:13,177

citizen developer friends of mine build, getting put together very differently than

Cue 17 — 00:01:13,217 --> 00:01:15,037

any software I've seen before.

Cue 18 — 00:01:15,457 --> 00:01:19,137

I have a friend who his whole company has run on a bunch of internal tools they

Cue 19 — 00:01:19,177 --> 00:01:21,417

built, and they're all hosted on Vercel.

Cue 20 — 00:01:21,457 --> 00:01:25,777

And I asked him why. He said, oh, because the authentication is at the network

Cue 21 — 00:01:25,817 --> 00:01:30,437

layer. We just log in and no one else outside the company can see it.

Cue 22 — 00:01:30,857 --> 00:01:33,497

And that was really interesting to me because like, auth is usually something inside

Cue 23 — 00:01:33,537 --> 00:01:36,117

of an app and people are moving it outside the app.

Cue 24 — 00:01:36,737 --> 00:01:41,877

And this is happening at OpenAI to a lot of the salespeople I work with have sent me

Cue 25 — 00:01:42,457 --> 00:01:46,177

sites. They're really great if you haven't used them, but they've built all these

Cue 26 — 00:01:46,217 --> 00:01:50,897

really complicated applications internally that I can see because I've logged in,

Cue 27 — 00:01:50,937 --> 00:01:52,917

but no one on the internet can see.

Cue 28 — 00:01:53,657 --> 00:01:57,737

And I think these citizen developers, like, oftentimes, with AI,

Cue 29 — 00:01:57,777 --> 00:02:01,457

have been living in the future and starting to do things and adopt certain patterns

Cue 30 — 00:02:01,497 --> 00:02:05,737

that I think the rest of us in the software industry are still not like all the way

Cue 31 — 00:02:05,777 --> 00:02:09,477

embracing and haven't really seen come for regular app development.

Cue 32 — 00:02:11,217 --> 00:02:15,177

My second observation, I was, you know, in February or March,

Cue 33 — 00:02:15,217 --> 00:02:19,417

like very confident that as soon as we had cloud agents and dev environments that

Cue 34 — 00:02:19,457 --> 00:02:22,837

everyone would basically have a software factory within a month or two.

Cue 35 — 00:02:23,297 --> 00:02:27,457

The Ramp blog post came out. We were all talking about how all the development is

Cue 36 — 00:02:27,497 --> 00:02:31,097

going to move to the cloud. And what's been interesting to me is like while some of

Cue 37 — 00:02:31,137 --> 00:02:34,657

that has happened and there's definitely a lot of people using cloud agents today.

Cue 38 — 00:02:34,697 --> 00:02:38,177

I still don't feel like you have an out of the box software factory,

Cue 39 — 00:02:38,217 --> 00:02:41,857

and a lot of these sort of promises and hopes have actually been really hard to

Cue 40 — 00:02:41,897 --> 00:02:44,057

manifest inside of real companies.

Cue 41 — 00:02:44,097 --> 00:02:46,137

And that's been a theme today.

Cue 42 — 00:02:46,177 --> 00:02:50,837

Like there's a lot of great work happening, but this is not like turnkey for anyone.

Cue 43 — 00:02:52,537 --> 00:02:56,957

And then observation number three, like code review is still a huge bottleneck.

Cue 44 — 00:02:57,417 --> 00:03:00,277

There's certain companies that are trying to auto merge things.

Cue 45 — 00:03:00,657 --> 00:03:05,417

There's obviously a lot of work going into figuring out what review agents should

Cue 46 — 00:03:05,457 --> 00:03:09,777

look like. But I think what's really changed here that, you know,

Cue 47 — 00:03:09,817 --> 00:03:13,657

no one's really talked about like, explicitly is that when you wrote code before,

Cue 48 — 00:03:13,697 --> 00:03:16,217

you were also testing it. You were also reviewing it.

Cue 49 — 00:03:16,257 --> 00:03:19,377

It was just like loop where you would do something and then verify it,

Cue 50 — 00:03:19,417 --> 00:03:21,497

and then do another thing and then verify it.

Cue 51 — 00:03:21,537 --> 00:03:26,257

And then when I saw a pull request, I at least knew that you had tested it all the

Cue 52 — 00:03:26,297 --> 00:03:29,617

way, and that you had tried everything and were relatively confident that it at

Cue 53 — 00:03:29,657 --> 00:03:31,457

least worked. And that's different now.

Cue 54 — 00:03:31,497 --> 00:03:34,097

Like sometimes I see a pull request get open and I'm like, I have to,

Cue 55 — 00:03:34,137 --> 00:03:37,137

you know, go all the way back to zero and just figure out, does this do the job that

Cue 56 — 00:03:37,177 --> 00:03:41,537

it's supposed to do? So in some ways this has become like more difficult and the

Cue 57 — 00:03:41,577 --> 00:03:45,637

verification has become something that you have to have the computer do all of,

Cue 58 — 00:03:45,897 --> 00:03:49,977

or the agent do all of, instead of a person doing like some of it and feeling

Cue 59 — 00:03:50,017 --> 00:03:54,677

confident that that trust between people could mediate a lot of the review concerns.

Cue 60 — 00:03:54,737 --> 00:03:56,697

So this stuff is sort of what's changed.

Cue 61 — 00:03:56,737 --> 00:04:01,657

I think this year, like what's been sort of standing out to me and as we all go and

Cue 62 — 00:04:01,697 --> 00:04:04,697

start trying to build these software factories that thought it'd be good for a

Cue 63 — 00:04:04,737 --> 00:04:07,877

second to sort of step back and think about what are the actual goals.

Cue 64 — 00:04:08,337 --> 00:04:11,297

So I talked to a lot of CTOs. I talked to a lot of platform teams,

Cue 65 — 00:04:11,337 --> 00:04:13,997

and this is what I hear kind of over and over again.

Cue 66 — 00:04:14,417 --> 00:04:17,717

The first thing is everyone wants to be faster to market.

Cue 67 — 00:04:18,937 --> 00:04:23,377

People also want to do more. So this is like, take those bigger bets that maybe if

Cue 68 — 00:04:23,417 --> 00:04:26,897

you had to spin up a team or invest a lot of time or money in,

Cue 69 — 00:04:26,937 --> 00:04:29,017

you wouldn't necessarily do AI.

Cue 70 — 00:04:29,057 --> 00:04:31,597

Hopefully you can let us try new things and learn faster.

Cue 71 — 00:04:32,457 --> 00:04:37,817

There's also this desire on a lot of teams to allow non-technical people to do

Cue 72 — 00:04:37,857 --> 00:04:42,657

meaningful work. So this is like opening PRs, designers, being able to contribute

Cue 73 — 00:04:42,697 --> 00:04:45,957

code or rebuild a page in the app.

Cue 74 — 00:04:46,417 --> 00:04:48,457

And we have to figure out how to do this safely.

Cue 75 — 00:04:48,497 --> 00:04:51,337

And that's honestly one of the biggest challenges with bringing these people into

Cue 76 — 00:04:51,377 --> 00:04:53,597

the into the software development lifecycle.

Cue 77 — 00:04:54,577 --> 00:04:57,397

And then we also want to give agents a lot of the groundwork,

Cue 78 — 00:04:57,697 --> 00:05:01,897

patches, updates, all the maintenance stuff that we don't really want to do but have

Cue 79 — 00:05:01,937 --> 00:05:07,057

to get done. This is also super important, and it requires all of the verification

Cue 80 — 00:05:07,097 --> 00:05:10,617

loops and all the autonomy. That is sort of hard to grant today.

Cue 81 — 00:05:10,657 --> 00:05:12,557

Not everyone's figured out how to do this yet.

Cue 82 — 00:05:13,737 --> 00:05:18,297

So, as I've been thinking about this problem a lot, I realized that a lot of the

Cue 83 — 00:05:18,337 --> 00:05:21,777

effort sort of this year has been going to dev infra.

Cue 84 — 00:05:21,817 --> 00:05:26,217

So these are things like cloud agents, dev environments, building code review agents

Cue 85 — 00:05:26,257 --> 00:05:31,757

and incident response agents. All the little at mention and assign things in Slack.

Cue 86 — 00:05:32,017 --> 00:05:35,937

But I think what we're not spending as much time on and probably should be is stuff

Cue 87 — 00:05:35,977 --> 00:05:42,397

on the app platform side. So building more libraries, better design systems,

Cue 88 — 00:05:43,137 --> 00:05:48,237

you know, data and query things, centralized auth primitives,

Cue 89 — 00:05:48,697 --> 00:05:52,377

things for audit logging, all of the stuff that maybe was just part of your code

Cue 90 — 00:05:52,417 --> 00:05:55,897

before. Now could potentially be platformized.

Cue 91 — 00:05:55,937 --> 00:05:57,957

I think there's a lot of benefits for doing this.

Cue 92 — 00:05:59,177 --> 00:06:03,057

When we think about dev infra, it's very much like, how do these agents run?

Cue 93 — 00:06:03,097 --> 00:06:06,257

But when we think about the app platform side, it's like, what are these agents

Cue 94 — 00:06:06,297 --> 00:06:10,017

doing? How are they building? What Lego blocks are they sort of assembling into our

Cue 95 — 00:06:10,057 --> 00:06:11,237

applications?

Cue 96 — 00:06:12,977 --> 00:06:14,957

So I think the math has really changed.

Cue 97 — 00:06:15,417 --> 00:06:20,277

We can afford to build abstractions that we used to skip.

Cue 98 — 00:06:20,657 --> 00:06:24,217

It was always very expensive. Like, do we build a library here?

Cue 99 — 00:06:24,257 --> 00:06:26,837

Do we just leave the code sort of in line?

Cue 100 — 00:06:27,497 --> 00:06:30,497

You know, do we know enough to actually try to abstract this yet?

Cue 101 — 00:06:30,537 --> 00:06:33,557

Or does it take a lot more time before we'll be able to do this?

Cue 102 — 00:06:33,897 --> 00:06:37,497

I think the cool thing about agents is you don't really have to make the same trade

Cue 103 — 00:06:37,537 --> 00:06:41,757

offs anymore. If you learn something new and you need to change the abstraction.

Cue 104 — 00:06:42,057 --> 00:06:44,677

You can just have the agents go update all the call sites.

Cue 105 — 00:06:45,057 --> 00:06:49,017

And I think there's this interesting thing about starting to build more of these

Cue 106 — 00:06:49,057 --> 00:06:51,937

building blocks that wasn't really accessible before.

Cue 107 — 00:06:51,977 --> 00:06:54,037

And more and more it is today.

Cue 108 — 00:06:55,177 --> 00:06:58,797

So, this is a I've been sort of trying to build.

Cue 109 — 00:06:58,937 --> 00:07:02,977

Instead of just giving it a prompt and saying, go brrr and use your tokens.

Cue 110 — 00:07:03,017 --> 00:07:07,277

I've been trying to have it actually build, all of the pieces first.

Cue 111 — 00:07:07,537 --> 00:07:11,057

So I was working on a dashboard the other day, and I had it like,

Cue 112 — 00:07:11,097 --> 00:07:16,297

go build a query layer first, then the chart library, and only then did I ask it,

Cue 113 — 00:07:16,337 --> 00:07:20,777

hey, go build the dashboard. And I tried to spend a lot of time like figuring out

Cue 114 — 00:07:20,817 --> 00:07:24,777

the shape of each of these components so that when I wanted to go extend the app and

Cue 115 — 00:07:24,817 --> 00:07:29,237

build more things, I could feel more confident in the foundation that I'd set.

Cue 116 — 00:07:31,297 --> 00:07:36,217

So, so going back to some of these goals, I think spending a lot of time building in

Cue 117 — 00:07:36,257 --> 00:07:40,537

the app platform can really open the doors and make software factories more

Cue 118 — 00:07:40,577 --> 00:07:47,177

effective. And the big reason is, again, you can go to market faster if you have a

Cue 119 — 00:07:47,217 --> 00:07:50,297

lot of pieces and you don't need to spend as much time reviewing,

Cue 120 — 00:07:50,337 --> 00:07:54,617

like all this inline code and can and can look at sort of smaller pull requests that

Cue 121 — 00:07:54,657 --> 00:07:57,817

are easier and more scoped, that are built on platform primitives that you've

Cue 122 — 00:07:57,857 --> 00:07:59,677

already tested and centralized.

Cue 123 — 00:08:00,337 --> 00:08:03,217

You know, this is sort of the dream to go faster.

Cue 124 — 00:08:03,257 --> 00:08:06,617

And I think using some of these shared pieces can help you feel a lot more confident

Cue 125 — 00:08:06,657 --> 00:08:08,037

about shipping things.

Cue 126 — 00:08:09,617 --> 00:08:11,977

I think it also helps you do more and take more bets.

Cue 127 — 00:08:12,017 --> 00:08:16,097

So obviously you can slop cannon out a lot of code.

Cue 128 — 00:08:16,137 --> 00:08:19,837

But if you spend your time building some of these central primitives,

Cue 129 — 00:08:19,937 --> 00:08:23,937

it does cost more upfront. But then the maintenance cost is a lot easier over time.

Cue 130 — 00:08:23,977 --> 00:08:27,317

And if you want to put out a lot more experiments and try things,

Cue 131 — 00:08:27,457 --> 00:08:30,617

you can be more confident that you're not going to also spend a lot more time

Cue 132 — 00:08:30,657 --> 00:08:33,997

maintaining those things, finding novel bugs in them.

Cue 133 — 00:08:34,337 --> 00:08:39,017

Hopefully doing a good job of building platform actually helps you expand the reach

Cue 134 — 00:08:39,057 --> 00:08:40,317

of what you're able to do.

Cue 135 — 00:08:42,377 --> 00:08:44,097

I think there's also some evidence.

Cue 136 — 00:08:44,137 --> 00:08:46,017

You know, oftentimes it's like, what is big tech doing?

Cue 137 — 00:08:46,057 --> 00:08:47,997

And everyone else has it ten years later.

Cue 138 — 00:08:48,377 --> 00:08:52,177

You see, like all of these, you know, very big internal platforms,

Cue 139 — 00:08:52,217 --> 00:08:55,477

they get built and have them built over the last couple of years.

Cue 140 — 00:08:55,697 --> 00:09:00,757

Zanzibar is like the famous Google auth service that basically does RBAC for

Cue 141 — 00:09:00,797 --> 00:09:05,197

everything in the Google ecosystem in like milliseconds, and is very effective.

Cue 142 — 00:09:05,497 --> 00:09:10,897

Meta has TAO. Uber has their machine learning platform that all the teams doing ML

Cue 143 — 00:09:10,937 --> 00:09:16,497

Uber use. Kafka came out of LinkedIn, because they were trying to figure out how to

Cue 144 — 00:09:16,537 --> 00:09:20,897

build queues. So a lot of these platforms have been built over the last couple of

Cue 145 — 00:09:20,937 --> 00:09:23,137

years. You know, decades even.

Cue 146 — 00:09:23,177 --> 00:09:26,337

And I think more and more will find that building internal platforms is like,

Cue 147 — 00:09:26,377 --> 00:09:29,197

accessible to companies where it was never accessible before.

Cue 159 — 00:10:11,152 --> 00:10:13,610

also want to enable non-technical roles.

Cue 160 — 00:10:13,650 --> 00:10:20,170

So, PMs, designers and other folks in a company to

Cue 161 — 00:10:20,210 --> 00:10:24,850

contribute code. And I think this is actually an area where thinking in platforms

Cue 162 — 00:10:24,890 --> 00:10:29,970

becomes very, very important. I don't know about you guys, but I don't necessarily

Cue 163 — 00:10:30,010 --> 00:10:35,730

want, the designer to be able to, you know, generate new queries or build new roles

Cue 164 — 00:10:35,770 --> 00:10:40,050

in our system or set up durable workflows or any of these things.

Cue 165 — 00:10:40,090 --> 00:10:43,250

I think there's a lot of stuff I'd want the designer to be able to do in the app,

Cue 166 — 00:10:43,290 --> 00:10:48,370

but I don't really have like clear boundaries today, especially when the agents just

Cue 167 — 00:10:48,410 --> 00:10:52,590

like to write all their code basically in line without componentizing anything.

Cue 168 — 00:10:52,730 --> 00:10:58,190

So I could imagine a world where as people build more of these primitives,

Cue 169 — 00:10:58,530 --> 00:11:03,290

you can end up in a in a situation where like there's a very clear sort of like zone

Cue 170 — 00:11:03,330 --> 00:11:07,390

of engagement where non-technical people can contribute code.

Cue 171 — 00:11:07,930 --> 00:11:11,410

But it's all dependent on these, you know, platform components that they import and

Cue 172 — 00:11:11,450 --> 00:11:15,110

use. And that makes everything, I think, a lot easier to scale.

Cue 173 — 00:11:15,530 --> 00:11:19,310

You can be a lot more confident that if they're editing something on the front end,

Cue 174 — 00:11:19,850 --> 00:11:24,450

maybe your code review rules make it so, you know, escalates to a much more serious

Cue 175 — 00:11:24,490 --> 00:11:28,930

review. If any of that code from that person, you know, touches something outside of

Cue 176 — 00:11:28,970 --> 00:11:32,410

that box. So I think there's a lot of interesting things here.

Cue 177 — 00:11:32,450 --> 00:11:37,190

If we want to bring in more of these people to building software with us.

Cue 178 — 00:11:39,210 --> 00:11:41,390

And the last thing is the grunt work.

Cue 179 — 00:11:41,850 --> 00:11:46,610

Obviously, it's been a very crazy last couple of weeks with the security stuff.

Cue 180 — 00:11:46,650 --> 00:11:50,110

A lot of teams are rushing very quickly to patch everything.

Cue 181 — 00:11:50,770 --> 00:11:55,290

And I think it's a lot easier to make these patches when, things are more

Cue 182 — 00:11:55,330 --> 00:12:00,430

centralized. So if you have, you know, better primitives, better platforms,

Cue 183 — 00:12:00,490 --> 00:12:04,230

it's a lot easier, I think, to change things in a few select areas,

Cue 184 — 00:12:04,530 --> 00:12:08,950

enforce sort of the contract of of those different parts of the system,

Cue 185 — 00:12:09,210 --> 00:12:13,810

get confidence that like, this piece still works the way it's supposed to work.

Cue 186 — 00:12:13,850 --> 00:12:18,170

And then, as it's used across the rest of your app boundaries,

Cue 187 — 00:12:18,210 --> 00:12:22,290

like it can actually hopefully like make it easier to propagate those fixes to more

Cue 188 — 00:12:22,330 --> 00:12:23,190

places.

Cue 189 — 00:12:25,970 --> 00:12:29,030

Moving back, like just to the big picture here.

Cue 190 — 00:12:29,650 --> 00:12:34,010

I don't know if anyone remembers, like, that famous API mandate from Amazon.

Cue 191 — 00:12:34,050 --> 00:12:36,810

It's kind of crazy. It was like more than two decades ago.

Cue 192 — 00:12:36,850 --> 00:12:39,370

Does anyone remember this? All right.

Cue 193 — 00:12:39,410 --> 00:12:43,610

Got a couple. So I think I think this may have been, like, early,

Cue 194 — 00:12:43,650 --> 00:12:49,970

but definitely not wrong. So this was the famous memo that, Bezos sent out to

Cue 195 — 00:12:50,010 --> 00:12:53,290

everyone at Amazon. It ended with, and if you don't, will fire you.

Cue 196 — 00:12:53,330 --> 00:12:58,790

But the beginning was expose data and functionality through APIs.

Cue 197 — 00:12:59,930 --> 00:13:03,350

All the teams communicate through those APIs.

Cue 198 — 00:13:04,450 --> 00:13:08,550

No direct database reads, shared memory or backdoors.

Cue 199 — 00:13:08,850 --> 00:13:12,450

I don't know about you, but I don't really want my agents writing arbitrary queries

Cue 200 — 00:13:12,490 --> 00:13:15,110

to my database and weird places in my code base.

Cue 201 — 00:13:16,330 --> 00:13:19,810

You can choose any technology you want behind the interface,

Cue 202 — 00:13:19,850 --> 00:13:24,530

and you have to design everything for external developers.

Cue 203 — 00:13:24,570 --> 00:13:26,510

I crossed that out and wrote agents.

Cue 204 — 00:13:27,010 --> 00:13:31,410

And I think this again, when I read this like ten years ago,

Cue 205 — 00:13:31,450 --> 00:13:34,190

it was like, yes, but I don't have time for this.

Cue 206 — 00:13:34,330 --> 00:13:39,250

And I think more and more as code has become cheap as it's become really easy to

Cue 207 — 00:13:39,290 --> 00:13:44,350

sort of refactor and things in your code base and compound some of the learnings

Cue 208 — 00:13:44,490 --> 00:13:48,850

about your domain. This is becoming, I think, not just like more possible,

Cue 209 — 00:13:48,890 --> 00:13:51,630

but also I think soon will be conventional wisdom.

Cue 210 — 00:13:52,170 --> 00:13:57,630

I would much rather build a castle out of Legos than out of sand.

Cue 211 — 00:13:57,850 --> 00:14:02,170

And I think that as we build more complex systems that we don't understand all the

Cue 212 — 00:14:02,210 --> 00:14:06,650

way, this will become super, super important because whether we like it or not,

Cue 213 — 00:14:06,690 --> 00:14:10,070

we are all sort of becoming these citizen developers.

Cue 214 — 00:14:10,410 --> 00:14:13,090

The agents are writing code that we don't understand.

Cue 215 — 00:14:13,130 --> 00:14:16,710

That's much, much more complicated than we could ever understand.

Cue 216 — 00:14:17,290 --> 00:14:21,570

And I think we're going to need to build systems that make it really easy for us to

Cue 217 — 00:14:21,610 --> 00:14:25,810

kind of do the right thing and contain sort of the units of work we're doing to like

Cue 218 — 00:14:25,850 --> 00:14:29,650

specific parts of an app so we can be more confident in the changes that we make and

Cue 219 — 00:14:29,690 --> 00:14:31,030

roll them out faster.

Cue 220 — 00:14:33,370 --> 00:14:38,130

There's I do think we need to be mindful of, the fact that, you know,

Cue 221 — 00:14:38,170 --> 00:14:41,170

bad abstractions are still a lot worse than no abstraction.

Cue 222 — 00:14:41,210 --> 00:14:43,870

So I've been experimenting with this a lot.

Cue 223 — 00:14:44,370 --> 00:14:47,410

Definitely encourage you guys to to be wary of that, though.

Cue 224 — 00:14:47,450 --> 00:14:49,970

Like, there is too far down this road.

Cue 225 — 00:14:50,010 --> 00:14:55,550

You could go. Also mindful of the fact that we don't have evals for this yet.

Cue 226 — 00:14:55,970 --> 00:14:58,810

And it's a really bad idea sometimes to work against the model.

Cue 227 — 00:14:58,850 --> 00:15:02,310

Like, the models really like to just spit out a ton of code.

Cue 228 — 00:15:02,730 --> 00:15:08,050

So figuring out the right way to, to build sort of these abstractions without losing

Cue 229 — 00:15:08,090 --> 00:15:09,810

performance, I think is really important.

Cue 230 — 00:15:09,850 --> 00:15:13,150

I'm trying to build some evals for this and learn more about it.

Cue 231 — 00:15:13,490 --> 00:15:17,470

But it's still very early, so I'll give you guys that caveat.

Cue 232 — 00:15:18,490 --> 00:15:21,570

You also are going to need to use skills to coax the models into using these

Cue 233 — 00:15:21,610 --> 00:15:25,230

components. Like if they haven't found them themselves, they definitely won't.

Cue 234 — 00:15:25,770 --> 00:15:30,170

And then just like a lot of people have to say no to their agents when they want to

Cue 235 — 00:15:30,210 --> 00:15:34,030

go update the test suite and make it all pass.

Cue 236 — 00:15:34,290 --> 00:15:35,810

You might have to say no to the agents a lot.

Cue 237 — 00:15:35,850 --> 00:15:39,430

If they want to go update one of these platform components that you build.

Cue 238 — 00:15:41,690 --> 00:15:45,150

So the takeaway I think it's important to experiment here.

Cue 239 — 00:15:45,530 --> 00:15:48,810

Again, there's a lot of great work happening on the dev infra side as we build out

Cue 240 — 00:15:48,850 --> 00:15:54,010

our software factories. I think we should be spending as much time exploring what it

Cue 241 — 00:15:54,050 --> 00:15:57,810

would mean to build more app platforms internally, and how that could enable us to

Cue 242 — 00:15:57,850 --> 00:16:01,170

bring more people into software development in our companies,

Cue 243 — 00:16:01,210 --> 00:16:05,850

make our apps more secure, and also help us move faster and try out a lot more

Cue 244 — 00:16:05,890 --> 00:16:11,174

things. So my second alt title here is: maybe the software factory is just the

Cue 245 — 00:16:11,214 --> 00:16:13,110

interfaces we made along the way.

Cue 246 — 00:16:13,850 --> 00:16:15,210

Cool. Well, thank you so much.

Cue 247 — 00:16:15,250 --> 00:16:18,230

Got a few minutes for questions and appreciate you all being here today.

Cue 248 — 00:16:24,050 --> 00:16:27,170

We have a few minutes for questions, so if you do have a question,

Cue 249 — 00:16:27,210 --> 00:16:29,670

please raise your hand and I'll get a mic to you.

Cue 250 — 00:16:36,050 --> 00:16:37,870

How do you think about testing?

Cue 251 — 00:16:41,250 --> 00:16:44,970

How do I think about testing? I think testing is super important.

Cue 252 — 00:16:45,010 --> 00:16:49,390

I think generally I've never really seen an agent test suite that I liked.

Cue 253 — 00:16:49,770 --> 00:16:55,130

So I often recommend and also try to, like, build them out, beforehand and then see

Cue 254 — 00:16:55,170 --> 00:16:56,850

if the agents will, will write to them.

Cue 255 — 00:16:56,890 --> 00:16:58,570

Sort of test driven development style.

Cue 256 — 00:16:58,610 --> 00:17:01,590

They don't like that. And it's really hard to get them to do it.

Cue 257 — 00:17:01,970 --> 00:17:05,130

But I do think, like being clear about what you want your, your software to do as

Cue 258 — 00:17:05,170 --> 00:17:10,050

super important. And the agents just literally always write test that pass,

Cue 259 — 00:17:10,090 --> 00:17:13,070

even if they shouldn't, sometimes egregiously.

Cue 260 — 00:17:20,870 --> 00:17:23,490

How would I go about platforming a design system?

Cue 261 — 00:17:24,230 --> 00:17:27,830

So I've seen a lot of, like, really cool stuff internally.

Cue 262 — 00:17:27,870 --> 00:17:33,150

We've done around this. Especially for someone like the interactive UIs inside of

Cue 263 — 00:17:33,190 --> 00:17:38,670

chat. So those are all like tokenized design system components that the agents can

Cue 264 — 00:17:38,710 --> 00:17:43,210

sort of write to. There's also obviously been a lot of great UI libraries people

Cue 265 — 00:17:43,250 --> 00:17:46,990

have built for a long time. And I think, again, it's like a really hard thing for a

Cue 266 — 00:17:47,030 --> 00:17:50,330

design team to go get that built without engineering resources.

Cue 267 — 00:17:50,750 --> 00:17:53,270

Now they can and now it's a lot cheaper to actually maintain that.

Cue 268 — 00:17:53,310 --> 00:17:56,710

So building those internally I think is one of the best places to start doing

Cue 269 — 00:17:56,750 --> 00:18:00,630

something like this. Oh, hi. I also have a question.

Cue 270 — 00:18:00,670 --> 00:18:04,710

So you mentioned a lot about infra versus app platform.

Cue 271 — 00:18:04,750 --> 00:18:08,350

So in this case what was your favorite spot on the dev infra side.

Cue 272 — 00:18:08,390 --> 00:18:11,670

Given that there are so many like production ready agents available.

Cue 273 — 00:18:11,710 --> 00:18:14,650

Like what is your daily use in this case?

Cue 274 — 00:18:15,070 --> 00:18:19,990

Yeah I, I think, you know, generally there's a lot of different ways to put things

Cue 275 — 00:18:20,030 --> 00:18:22,770

in a container and have it run in the cloud.

Cue 276 — 00:18:23,270 --> 00:18:25,990

So really it's like, what do you actually have?

Cue 277 — 00:18:26,030 --> 00:18:30,030

Have those things do. Like, do you, do you want to fill it with skills that help it

Cue 278 — 00:18:30,070 --> 00:18:34,430

do certain kinds of tasks? Do you want it to be doing like the full testing loop and

Cue 279 — 00:18:34,470 --> 00:18:35,840

actually like trying to verify the software?

Cue 280 — 00:18:35,880 --> 00:18:38,030

Or do you want to open the PR and then have another one go review it?

Cue 281 — 00:18:38,070 --> 00:18:42,710

Like there's a lot of, you know, decisions people make when they build out some of

Cue 282 — 00:18:42,750 --> 00:18:46,950

that infrastructure. And I don't have like I don't have strong recommendations.

Cue 283 — 00:18:46,990 --> 00:18:49,930

I think I think it's just going to be finding what works well for your team.

Cue 284 — 00:18:52,110 --> 00:18:56,337

There's a bit of a friction growing between the citizen dev vibe coders,

Cue 285 — 00:18:56,377 --> 00:19:01,010

shadow IT, whatever you want to call them, and who have to follow the whole

Cue 286 — 00:19:01,230 --> 00:19:07,798

pipeline. The as you said, they go and that goes much faster than what IT,

Cue 287 — 00:19:07,838 --> 00:19:11,950

the devs are going to do. So what I would pick the number three.

Cue 288 — 00:19:11,990 --> 00:19:16,810

What are your top three rules that you don't allow the vibe coders to skip?

Cue 289 — 00:19:17,670 --> 00:19:22,470

Yeah, I think the vibe coders are always going to find a way to ship,

Cue 290 — 00:19:22,510 --> 00:19:24,810

so it is better to have them ship in a box.

Cue 291 — 00:19:24,990 --> 00:19:30,070

And in a place you control. So I think absolutely like whatever they deploy on,

Cue 292 — 00:19:30,110 --> 00:19:34,830

whether it's like Sites or Vercel or some internal infrastructure people build that

Cue 293 — 00:19:34,870 --> 00:19:37,240

should not have external network access and that should have,

Cue 294 — 00:19:37,280 --> 00:19:40,710

like, auth literally at the network layer where people have to like log in to get to

Cue 295 — 00:19:40,750 --> 00:19:45,950

the, the resources. I think you also need to be really mindful of like what data

Cue 296 — 00:19:45,990 --> 00:19:50,190

those things have access to. So if it's like a read only MCP versus a write MCP,

Cue 297 — 00:19:50,230 --> 00:19:54,510

and then also recognize like anyone who can log into this vibe coded app can have

Cue 298 — 00:19:54,550 --> 00:19:57,970

those permissions too. So like that is I think a huge problem.

Cue 299 — 00:19:58,150 --> 00:20:02,350

Probably nondestructive non-write actions are like what you should grant to those by

Cue 300 — 00:20:02,390 --> 00:20:03,290

default.

Cue 301 — 00:20:04,710 --> 00:20:05,990

And I'm just going to stick to two rules.

Cue 302 — 00:20:06,030 --> 00:20:08,130

I think that's what was most important here.

Cue 303 — 00:20:09,710 --> 00:20:12,790

I have a question about testing again.

Cue 304 — 00:20:12,830 --> 00:20:16,030

And it's mostly about the different types of testing.

Cue 305 — 00:20:16,070 --> 00:20:17,270

And what do you think about them?

Cue 306 — 00:20:17,310 --> 00:20:22,830

Like I feel like AI, it's wonderful making a ton of shitty unit test.

Cue 307 — 00:20:22,870 --> 00:20:28,350

And then you figure out you have like 200K lines of code and like 500K unit tests.

Cue 308 — 00:20:28,390 --> 00:20:31,790

So my question is like, what do you think are good test, for example,

Cue 309 — 00:20:31,830 --> 00:20:35,630

integration, mutation testing or other kinds?

Cue 310 — 00:20:35,670 --> 00:20:40,610

And which ones do you think are like just shitty stuff that AI tends to do?

Cue 311 — 00:20:41,110 --> 00:20:45,890

I think by default I just assume that they're all shitty tests that are sloppy.

Cue 312 — 00:20:46,430 --> 00:20:49,790

And I try to be really clear about, like, what I wanted to test.

Cue 313 — 00:20:49,830 --> 00:20:54,430

So, like, actually writing a test plan, I think is, is like a worthwhile exercise

Cue 314 — 00:20:54,470 --> 00:20:57,750

when you're prompting. And even once you do that, they will all still write the

Cue 315 — 00:20:57,790 --> 00:20:59,350

other tests that you don't want to.

Cue 316 — 00:20:59,390 --> 00:21:01,850

So just making sure to, like, prune the garden.

Cue 317 — 00:21:02,070 --> 00:21:06,250

But I think I think again, if you like, are building smaller pieces of software,

Cue 318 — 00:21:06,510 --> 00:21:09,350

they also become easier to test and like integration test.

Cue 319 — 00:21:09,390 --> 00:21:14,410

Someone made a great graph once of like how the test cases compound like the more

Cue 320 — 00:21:14,470 --> 00:21:15,990

you're trying to test at the same time.

Cue 321 — 00:21:16,030 --> 00:21:21,510

So I think smaller, more component style APIs, and other services and libraries

Cue 322 — 00:21:21,550 --> 00:21:23,290

actually make the problem easier to.

Cue 323 — 00:21:24,350 --> 00:21:28,110

Yeah. So you hit on at the end the question of enforcement of component usage,

Cue 324 — 00:21:28,150 --> 00:21:29,430

which I think is really interesting.

Cue 325 — 00:21:29,470 --> 00:21:31,230

Right. It's like, yes, put it in skills.

Cue 326 — 00:21:31,270 --> 00:21:35,390

Yes. You have to mash the no button a lot, but like that doesn't scale in a large

Cue 327 — 00:21:35,430 --> 00:21:40,530

organization. Is there a role for like deterministic tooling to play in enforcing a

Cue 328 — 00:21:40,990 --> 00:21:43,910

usage of these components, or is this constantly going to be a race that we're

Cue 329 — 00:21:43,950 --> 00:21:47,670

running? Yeah, I think it's a I think it's a good question.

Cue 330 — 00:21:47,710 --> 00:21:50,010

I'm also hopeful that like,

Cue 331 — 00:21:51,510 --> 00:21:57,110

it's going to get easier to like if we are able to get the models a lot better at

Cue 332 — 00:21:57,150 --> 00:22:01,390

like using things that aren't in their distribution, like, so like libraries and

Cue 333 — 00:22:01,430 --> 00:22:03,150

your own APIs. And these things like are not.

Cue 334 — 00:22:03,190 --> 00:22:07,030

So if they were trained on, maybe they're, they're better at picking those things up

Cue 335 — 00:22:07,070 --> 00:22:11,530

when they see them. But yeah, I think, I think it's probably going to be a mix of

Cue 336 — 00:22:12,270 --> 00:22:17,670

like develop time things like rules and skills and then also verification type

Cue 337 — 00:22:17,710 --> 00:22:20,490

things in code review being like, no, you have to do it this way.

Cue 338 — 00:22:20,790 --> 00:22:24,070

And it's obviously so much more expensive when you catch it late because then you

Cue 339 — 00:22:24,110 --> 00:22:27,410

have to go run it again and, you know, all that stuff.

Cue 340 — 00:22:27,550 --> 00:22:31,230

But I do think it is a little bit of a race and we're all sort of figuring this out.

Cue 341 — 00:22:31,270 --> 00:22:33,590

It's crazy just how much has changed in a year.

Cue 342 — 00:22:33,630 --> 00:22:37,970

And, this talk is definitely a more heady like where it might be next year.

Cue 343 — 00:22:38,230 --> 00:22:41,370

And I think that's one of the biggest questions we have to figure out as we build.

Cue 344 — 00:22:43,350 --> 00:22:49,790

Okay. What's your existing workflow, if any, for using kind of production level

Cue 345 — 00:22:49,830 --> 00:22:54,070

traces or just seeing what's out there in production to kind of like,

Cue 346 — 00:22:54,110 --> 00:22:57,770

strengthen the existing interfaces you have, like add new ones.

Cue 347 — 00:22:59,270 --> 00:23:03,010

I haven't done a ton of stuff with, with live logs.

Cue 348 — 00:23:03,590 --> 00:23:09,230

But I, I guess my, my beliefs about it is that we're going to want to have some kind

Cue 349 — 00:23:09,270 --> 00:23:13,510

of persistence around, like the prompts that we use to generate different parts of

Cue 350 — 00:23:13,550 --> 00:23:18,190

the code base. Those should know about, like the logging events that are there and

Cue 351 — 00:23:18,230 --> 00:23:20,910

then should somehow be able to correlate all those signals.

Cue 352 — 00:23:20,950 --> 00:23:26,030

So you can go like talk to the agent who wrote that code about the traces that are

Cue 353 — 00:23:26,070 --> 00:23:27,770

relevant to the thing it wrote.

Cue 354 — 00:23:28,550 --> 00:23:30,310

These are the sort of magical things that you're like.

Cue 355 — 00:23:30,350 --> 00:23:33,190

All of the data is there, and it should all be possible.

Cue 356 — 00:23:33,230 --> 00:23:35,110

But like, no one has made it easy yet.

Cue 357 — 00:23:35,150 --> 00:23:38,790

And I'd really love to see, like, more work from the infra and the observability

Cue 358 — 00:23:38,830 --> 00:23:40,050

tools around this.

Cue 359 — 00:23:41,950 --> 00:23:47,270

You mentioned keeping like PMs inside of a little contained area that's safe.

Cue 360 — 00:23:47,310 --> 00:23:51,630

And do you have any like working solutions for that?

Cue 361 — 00:23:51,670 --> 00:23:53,430

Like what is what are those apps like?

Cue 362 — 00:23:53,470 --> 00:23:57,190

For example, I keep some of my marketers inside of Claude Design.

Cue 363 — 00:23:57,230 --> 00:24:02,190

So the handoff is they give me a Claude Design artifact that's like interactive and

Cue 364 — 00:24:02,230 --> 00:24:05,390

looks how they want. And then we like integrate it from a development perspective.

Cue 365 — 00:24:05,430 --> 00:24:07,350

But that's like super limited obviously.

Cue 366 — 00:24:07,390 --> 00:24:11,390

Do you have like a more robust like set of tools that you do keep your PMs inside of

Cue 367 — 00:24:11,430 --> 00:24:16,730

right now? Great question. I think I'm going to start calling these things,

Cue 368 — 00:24:17,110 --> 00:24:19,890

sandboxes for humans instead of cages.

Cue 369 — 00:24:21,110 --> 00:24:26,390

And I think that, I think that that basically there's, there's a few ways to go

Cue 370 — 00:24:26,430 --> 00:24:31,470

about it. It's like for, for, I think it's easier for like, go to market people who

Cue 371 — 00:24:31,510 --> 00:24:34,070

have internal apps because all that stuff just going to get hosted,

Cue 372 — 00:24:34,110 --> 00:24:37,430

you know, and they're all disposable and kind of one off little apps for actually

Cue 373 — 00:24:37,470 --> 00:24:39,250

contributing to the code base.

Cue 374 — 00:24:39,790 --> 00:24:43,130

I think it'd be really cool if you had, like, skills that were,

Cue 375 — 00:24:43,590 --> 00:24:47,970

enabled inside of these agents, like, based on the role of the person,

Cue 376 — 00:24:48,270 --> 00:24:51,870

initiating it. I think that could be a really cool kind of approach to this.

Cue 377 — 00:24:51,910 --> 00:24:54,010

Again, no one has put out of the box.

Cue 378 — 00:24:54,390 --> 00:24:56,470

And then I think, again, it's at the verification time, too.

Cue 379 — 00:24:56,510 --> 00:25:02,710

Like if a PM code something and it's like updating the library that does auth.

Cue 380 — 00:25:02,750 --> 00:25:05,210

Like, I don't want that. Like, no.

Cue 381 — 00:25:05,750 --> 00:25:08,670

And maybe you block that, or maybe you just, like, ask someone to go review it for

Cue 382 — 00:25:08,710 --> 00:25:12,670

real. But I think, I think we're going to need to be reactive to kind of what these

Cue 383 — 00:25:12,710 --> 00:25:17,710

folks are doing. You mentioned RBAC and role-based security.

Cue 384 — 00:25:17,750 --> 00:25:21,790

A couple of times. I wonder if you think we should be moving to task based

Cue 385 — 00:25:21,830 --> 00:25:26,390

management in our world here and that role-based is just too coarse for where agents

Cue 386 — 00:25:26,430 --> 00:25:30,110

are concerned. Are probably for some applications.

Cue 387 — 00:25:30,150 --> 00:25:35,750

That would make sense. I think that the there's like these what should you do for

Cue 388 — 00:25:35,790 --> 00:25:39,870

your customers and then like what should you do for your developers who are building

Cue 389 — 00:25:39,910 --> 00:25:42,930

things? I think those those might look like really different.

Cue 390 — 00:25:43,230 --> 00:25:46,970

The customer one, you just you never want to sort of get it wrong.

Cue 391 — 00:25:47,710 --> 00:25:52,110

I think for the development one like usually development things are nondestructive.

Cue 392 — 00:25:52,150 --> 00:25:55,390

So if they, if they are able to update something into environment they shouldn't

Cue 393 — 00:25:55,430 --> 00:25:57,970

like, you can learn from that and make it better next time.

Cue 394 — 00:26:00,830 --> 00:26:06,130

If you have time for 1 or 2 more quick questions if anyone's got one.

Cue 395 — 00:26:11,530 --> 00:26:18,230

Yeah. About like the workflow of coding and like your specific architectures,

Cue 396 — 00:26:18,970 --> 00:26:24,330

what are your thoughts about speed of coding against architectures,

Cue 397 — 00:26:24,370 --> 00:26:26,850

against what agents should touch.

Cue 398 — 00:26:26,890 --> 00:26:30,930

And for example, you start something with normally domain driven design with

Cue 399 — 00:26:30,970 --> 00:26:35,570

something like screaming architecture from Uncle Bob and you end up with,

Cue 400 — 00:26:35,610 --> 00:26:40,450

again, a mess. How are you able to like kind of stop that while keeping the fast

Cue 401 — 00:26:40,490 --> 00:26:44,690

speed of AI agents? You ask you to like just the last part again.

Cue 402 — 00:26:44,730 --> 00:26:46,410

I missed the middle there. Yeah, yeah.

Cue 403 — 00:26:46,450 --> 00:26:49,490

Like you normally start a project with this huge idea of, like,

Cue 404 — 00:26:49,530 --> 00:26:53,370

I want this architecture and you're like, I will enforce this architecture.

Cue 405 — 00:26:53,410 --> 00:26:58,850

And at the half of the project, like the, the coupling of this is huge and the

Cue 406 — 00:26:58,890 --> 00:27:01,690

cohesion is low and it should be the opposite, like.

Cue 407 — 00:27:01,730 --> 00:27:04,050

How are you able to keep development speed.

Cue 408 — 00:27:04,090 --> 00:27:08,930

Because what I feel like from my opinion is that all these MD files I give to my

Cue 409 — 00:27:08,970 --> 00:27:10,490

agent, so like this is the architecture.

Cue 410 — 00:27:10,530 --> 00:27:12,850

Like go to have is like that's the road.

Cue 411 — 00:27:12,890 --> 00:27:17,370

But like they take whatever they want and I use all my unit tests,

Cue 412 — 00:27:17,410 --> 00:27:23,050

all my like integration tests, Bruno whatever whatsoever and looks good.

Cue 413 — 00:27:23,090 --> 00:27:26,130

And I don't have the time to go ahead and check every part of the architecture.

Cue 414 — 00:27:26,170 --> 00:27:31,870

So how are you able to be able to go fast without, like, losing and having that

Cue 415 — 00:27:32,010 --> 00:27:37,930

spaghetti code again. Yeah. No it's a it's a good question I think that you know

Cue 416 — 00:27:37,970 --> 00:27:42,010

done right, like building some of these reusable pieces and trying to keep,

Cue 417 — 00:27:42,050 --> 00:27:45,210

you know at least like the rails in place and trying to get them to run down the

Cue 418 — 00:27:45,250 --> 00:27:48,370

train is like a great I think it's better than the alternative of just like,

Cue 419 — 00:27:48,410 --> 00:27:50,110

go crazy, do whatever you want.

Cue 420 — 00:27:50,610 --> 00:27:54,250

But yeah, I still don't think they, they have the intuitions.

Cue 421 — 00:27:54,290 --> 00:27:58,850

I mean, hopefully, more work around memory and some of these other things that are

Cue 422 — 00:27:58,890 --> 00:28:01,710

coming together, like will have impact there.

Cue 423 — 00:28:02,130 --> 00:28:04,570

But I'd like them to have more like, core sense at some point.

Cue 424 — 00:28:04,610 --> 00:28:08,170

And they don't right now. And I kind of expect them to, and,

Cue 425 — 00:28:08,210 --> 00:28:11,490

and so building kind of like the right rails I think is still a,

Cue 426 — 00:28:11,530 --> 00:28:15,710

a good bet to make, even if they require a lot more coaxing than you want today.

Cue 427 — 00:28:16,970 --> 00:28:18,450

Cool. Well, thank you very much, everyone.

Cue 428 — 00:28:18,490 --> 00:28:19,770

Appreciate it. And thanks for the dialog.

Cue 429 — 00:28:19,810 --> 00:28:20,870

It's super fun.

skills

talk-cunniffe-reusable-agent-platform

tile.json