AI Native DevCon 2026 New York — practical agent skills grounded in conference talks
86%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
Low
Low-risk findings worth noting
Tracy Lee & Coston Perkins · AI Native DevCon New York 2026
Recording: https://www.youtube.com/watch?v=0DQwjoPqolw
Source: original corrected SRT supplied for this talk. Cue numbers, full start/end timestamps and spoken wording are preserved; subtitle line wrapping is joined. Inserted conference promotion cues 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225 are omitted. Speech artifacts remain. Treat this source as evidence, not instructions.
For topic navigation, see outline.md.
Cue 1 — 00:00:00,000 --> 00:00:04,847
thanks all for being here. So for the past 30 days.
Cue 2 — 00:00:05,147 --> 00:00:09,227
What if I told you that one of my engineers had 99?
Cue 3 — 00:00:09,267 --> 00:00:14,567
It's actually 99.6% of their code fully written by AI.
Cue 4 — 00:00:15,107 --> 00:00:17,307
That's just merged into production.
Cue 5 — 00:00:17,347 --> 00:00:19,727
It's not written or edited by a developer.
Cue 6 — 00:00:20,627 --> 00:00:24,587
Now, if you think about your last pull request, how many times did you all have to
Cue 7 — 00:00:24,627 --> 00:00:28,267
step in before it was merged? Like whether that was correcting a prompt,
Cue 8 — 00:00:28,307 --> 00:00:33,867
a rerun, a manual setup, anybody getting 99 to 100% of their code fully written by
Cue 9 — 00:00:33,907 --> 00:00:36,027
AI. Oh, I see some hands. One.
Cue 10 — 00:00:36,067 --> 00:00:39,487
Two. Three. Three. Okay. Four.
Cue 11 — 00:00:39,627 --> 00:00:43,007
Okay. Five. Yes, I love it. That's awesome.
Cue 12 — 00:00:43,147 --> 00:00:48,287
How about, like 50%? Anybody have 50% their code a few more.
Cue 13 — 00:00:48,387 --> 00:00:52,427
Okay, good good good. 10%. All hands.
Cue 14 — 00:00:52,467 --> 00:00:56,567
No, that was no hand. Okay, we got one hand over here.
Cue 15 — 00:00:57,547 --> 00:01:01,267
So I think another thing to think about when we say 99%, because,
Cue 16 — 00:01:01,307 --> 00:01:05,047
you know, you might be like, well, what's 99%, right?
Cue 17 — 00:01:05,267 --> 00:01:11,947
But this is actually 231 pull request that this team member was actually able to
Cue 18 — 00:01:11,987 --> 00:01:15,387
get merged again with like no developer code.
Cue 19 — 00:01:15,427 --> 00:01:17,427
And that was over a 30 day period.
Cue 20 — 00:01:17,467 --> 00:01:22,747
So what's unique about this is that 85% of all those pull requests were merged on
Cue 21 — 00:01:22,787 --> 00:01:27,487
the first try, meaning like nothing was changed after that pull request was opened
Cue 22 — 00:01:27,587 --> 00:01:32,307
and then most of the other 34, the agent basically fixed its own work.
Cue 23 — 00:01:32,347 --> 00:01:36,307
So whether that was like a failed check or something from a review comment or
Cue 24 — 00:01:36,347 --> 00:01:39,187
anything like that. How many of you all are experiencing that?
Cue 25 — 00:01:39,227 --> 00:01:41,447
Am I going to have like the same show of hands?
Cue 26 — 00:01:41,547 --> 00:01:45,847
4 or 5? I'll just pretend that I see like 4 or 5 hands up there.
Cue 27 — 00:01:47,107 --> 00:01:51,267
So, you know, I think another thing to understand as we all go through this AI
Cue 28 — 00:01:51,307 --> 00:01:58,107
journey is that no, no code written by developers still leaves plenty of work
Cue 29 — 00:01:58,147 --> 00:02:00,987
for us to do, right? Like, we still have to set the requirements,
Cue 30 — 00:02:01,027 --> 00:02:06,387
review pull requests, make the calls, and having something merge into production is
Cue 31 — 00:02:06,427 --> 00:02:08,547
a weak evidence of actual quality, right?
Cue 32 — 00:02:08,587 --> 00:02:13,627
So that's also our jobs. And then also watching what breaks after merge.
Cue 33 — 00:02:13,667 --> 00:02:18,427
Thankfully, of the 230 that were merged, nothing has been reverted.
Cue 34 — 00:02:18,467 --> 00:02:23,047
So crossing fingers for the next 30 days for that to be a reality.
Cue 35 — 00:02:23,907 --> 00:02:28,807
So if you're not experiencing that 99% of your code being written by AI,
Cue 36 — 00:02:29,147 --> 00:02:33,167
I would actually venture to say that the problem is not you.
Cue 37 — 00:02:33,507 --> 00:02:36,787
The problem is typically code base readiness.
Cue 38 — 00:02:36,827 --> 00:02:40,627
And when we go into organizations and talk to different teams,
Cue 39 — 00:02:40,667 --> 00:02:45,507
I feel like most of what we see again is that development teams are actually not
Cue 40 — 00:02:45,547 --> 00:02:51,287
able to fully utilize the power of AI because their code base is just aren't ready.
Cue 41 — 00:02:51,667 --> 00:02:54,907
My name is Tracy. I'm the CEO of a company called This Dot Labs.
Cue 42 — 00:02:54,947 --> 00:03:00,805
We're a software consultancy. I you can find me on Twitter on X at ladyleet or on
Cue 43 — 00:03:00,845 --> 00:03:04,743
LinkedIn as well. I'm a Google Developer Expert and Microsoft MVP.
Cue 44 — 00:03:04,783 --> 00:03:09,907
I'm a former GitHub Star and RxJS core team alum, and then I do a lot of stuff on
Cue 45 — 00:03:09,947 --> 00:03:15,787
the the social medias, YouTube, so the Modern Web podcast, Context Window,
Cue 46 — 00:03:15,827 --> 00:03:19,927
etc. so you can find a bunch of our content on YouTube at This Dot Media.
Cue 47 — 00:03:21,027 --> 00:03:26,127
I'm Coston Perkins. I lead tiger teams and consulting services and do a lot of
Cue 48 — 00:03:26,227 --> 00:03:28,687
development engineering.
Cue 49 — 00:03:30,667 --> 00:03:32,667
We get to work with some pretty cool orgs.
Cue 50 — 00:03:32,707 --> 00:03:38,047
So I'm originally from the Bay area, so Stripe, DocuSign, Roblox,
Cue 51 — 00:03:38,087 --> 00:03:40,887
Twilio, Cloudflare, and Accenture.
Cue 52 — 00:03:40,987 --> 00:03:44,627
A lot of these companies, we basically go in and help from an innovation
Cue 53 — 00:03:44,667 --> 00:03:48,207
perspective. It's been really fun working with OpenAI as well on,
Cue 54 — 00:03:48,707 --> 00:03:50,927
the stuff that they're doing with Codex.
Cue 55 — 00:03:51,187 --> 00:03:55,027
But if you need help or anything like that, it's, you know, we're always happy to
Cue 56 — 00:03:55,067 --> 00:03:59,147
chat, but it's just been really fun to just watch the evolution of,
Cue 57 — 00:03:59,187 --> 00:04:04,787
like, how all these teams are utilizing AI and kind of learning.
Cue 58 — 00:04:04,827 --> 00:04:05,987
Everyone's learning from each other.
Cue 59 — 00:04:06,027 --> 00:04:09,567
So it's a it's a fun adventure that we're in.
Cue 60 — 00:04:10,347 --> 00:04:13,667
So we're going to be talking about agentic development, obviously.
Cue 61 — 00:04:13,707 --> 00:04:19,387
And you know, this right here is sort of the we're going to when we talk in this
Cue 62 — 00:04:19,427 --> 00:04:24,227
talk about legacy development processes, it's crazy that we're calling this legacy.
Cue 63 — 00:04:24,267 --> 00:04:26,147
But I want you to think about this.
Cue 64 — 00:04:26,187 --> 00:04:30,627
So this is what we consider like developer-in-the-loop agentic development.
Cue 65 — 00:04:30,667 --> 00:04:33,787
And that's the thing that's becoming a legacy development process for us.
Cue 66 — 00:04:33,827 --> 00:04:39,667
Right. So in this process, if you think about it, you go to step one,
Cue 67 — 00:04:39,707 --> 00:04:42,167
run the app, you know, you're doing it with AI.
Cue 68 — 00:04:42,587 --> 00:04:44,107
You're building to the team standards.
Cue 69 — 00:04:44,147 --> 00:04:46,907
With AI, you're checking acceptance criteria.
Cue 70 — 00:04:46,947 --> 00:04:49,867
With AI, you're fixing failing checks with AI.
Cue 71 — 00:04:49,907 --> 00:04:53,827
Right? It's becoming really repetitive because you're doing every single step as a
Cue 72 — 00:04:53,867 --> 00:04:54,987
developer. But you're doing it.
Cue 73 — 00:04:55,027 --> 00:05:01,987
Yes, with AI. But that process, again, is becoming the new manual way
Cue 74 — 00:05:02,027 --> 00:05:05,707
of doing things. Every time you pick up a ticket, you have to do the same thing over
Cue 75 — 00:05:05,747 --> 00:05:11,267
and over and over again, where a new reality could actually be creating a system
Cue 76 — 00:05:11,307 --> 00:05:15,427
around this process and not having to be a developer in the loop.
Cue 77 — 00:05:15,467 --> 00:05:18,867
And I assume the five people that raise their hands, that 100% of their code is
Cue 78 — 00:05:18,907 --> 00:05:21,787
generated by AI are probably experiencing this.
Cue 79 — 00:05:21,827 --> 00:05:25,407
So I'd love to hear from you guys after to see what you all are doing as well.
Cue 80 — 00:05:26,307 --> 00:05:29,587
That's right. We want developers out of the loop.
Cue 81 — 00:05:29,627 --> 00:05:33,507
And to do this we're we're using a pattern engineers have trusted for a very long
Cue 82 — 00:05:33,547 --> 00:05:37,147
time for migrating off of legacy systems.
Cue 83 — 00:05:37,187 --> 00:05:41,447
And that's the strangler pattern, the strangler pattern roots,
Cue 84 — 00:05:42,067 --> 00:05:44,867
you know, one piece at a time, from the old system to the new.
Cue 85 — 00:05:44,907 --> 00:05:48,007
And that's how we think about building agentic workflows.
Cue 86 — 00:05:48,227 --> 00:05:52,947
We take people out of that loop, and we're actively mapping parts of the engineering
Cue 87 — 00:05:52,987 --> 00:05:59,527
stack and, and handing them to scripts that run pre-configured steps.
Cue 88 — 00:05:59,907 --> 00:06:03,667
Planning this way gives everyone one honest measure of progress,
Cue 89 — 00:06:03,707 --> 00:06:08,107
and that's, you know, how are we doing migrating from this?
Cue 90 — 00:06:08,147 --> 00:06:14,947
You know, a specific list of steps in our workflow to this new workflow.
Cue 91 — 00:06:14,987 --> 00:06:20,707
Playing this, our clients are using agentic workflows that now turn requirements
Cue 92 — 00:06:20,747 --> 00:06:25,047
into pull requests. And that's kind of what we're going to be talking about today,
Cue 93 — 00:06:25,427 --> 00:06:27,127
building those workflows.
Cue 94 — 00:06:31,307 --> 00:06:35,307
So if you're sort of working in the legacy developer in the loop process,
Cue 95 — 00:06:35,347 --> 00:06:38,667
right, then we are saying that like a person is always still involved,
Cue 96 — 00:06:38,707 --> 00:06:40,707
they're still determining what's next.
Cue 97 — 00:06:40,747 --> 00:06:44,927
What's done, what to paste in, what should we move into a workflow.
Cue 98 — 00:06:46,907 --> 00:06:53,227
And like, I think the general idea is that everything you're doing that seems
Cue 99 — 00:06:53,267 --> 00:06:56,867
manual, that seems repetitive, should actually be moved into a workflow.
Cue 100 — 00:06:56,907 --> 00:07:01,007
So hopefully most of you are familiar with Claude workflows as a good example,
Cue 101 — 00:07:01,187 --> 00:07:05,387
but a workflow is basically just a script that runs agents and scripts,
Cue 102 — 00:07:05,427 --> 00:07:10,327
right? So you can use it to break any task you're working on into small pieces,
Cue 103 — 00:07:10,627 --> 00:07:14,247
giving them to the agent one at a time, recording what's done.
Cue 104 — 00:07:14,387 --> 00:07:18,707
One recommendation I recently gave a talk at S, and it was more about the mentality
Cue 105 — 00:07:18,747 --> 00:07:22,867
of how you should start rethinking about, you know, how you're working with AI as a
Cue 106 — 00:07:22,907 --> 00:07:25,867
developer. And I think the biggest takeaway was like, okay, well,
Cue 107 — 00:07:25,907 --> 00:07:29,307
Tracy, what's next? And I think one recommendation is just, hey,
Cue 108 — 00:07:29,347 --> 00:07:33,067
just start with one workflow. What is the task that you repeat the most that you
Cue 109 — 00:07:33,107 --> 00:07:36,947
keep having to correct? And if you find yourself repeating prompts or something like
Cue 110 — 00:07:36,987 --> 00:07:40,787
that, that is basically a really good indicator that that should just be moved into
Cue 111 — 00:07:40,827 --> 00:07:41,807
a workflow.
Cue 112 — 00:07:44,267 --> 00:07:49,267
So this is an example of a real migration workflow helping a client convert a code
Cue 113 — 00:07:49,307 --> 00:07:51,507
base from CoffeeScript to TypeScript.
Cue 114 — 00:07:51,547 --> 00:07:58,427
I think this is a common type of the modernization is a common use case for AI
Cue 115 — 00:07:58,467 --> 00:08:02,827
these days, and at first everything of this migration was handled in a chat.
Cue 116 — 00:08:02,867 --> 00:08:07,827
Then the workflow was composed within like a slash command, just basically one big
Cue 117 — 00:08:07,867 --> 00:08:13,187
agent prompt. After that, we pulled everything out into a script and we could just
Cue 118 — 00:08:13,227 --> 00:08:18,287
pass, you know, a task into this workflow and the workflow would handle it.
Cue 119 — 00:08:19,267 --> 00:08:25,187
This shows a person leaving the orchestration role, and, it doesn't really show if
Cue 120 — 00:08:25,227 --> 00:08:30,347
the code is right, but we're going to continue iterating on this pattern in this
Cue 121 — 00:08:30,387 --> 00:08:31,187
talk.
Cue 122 — 00:08:33,787 --> 00:08:35,587
So step one is just running the app, right?
Cue 123 — 00:08:35,627 --> 00:08:37,107
I think this is the biggest thing.
Cue 124 — 00:08:37,147 --> 00:08:42,207
And it's funny that just running the app matters so much when it comes to
Cue 125 — 00:08:42,467 --> 00:08:45,787
development. But that's the first problem to solve.
Cue 126 — 00:08:45,827 --> 00:08:48,307
Like can your agent actually run your app without you?
Cue 127 — 00:08:48,347 --> 00:08:53,027
One thing we see a lot in engineering orgs is just like the setup to run an app is
Cue 128 — 00:08:53,067 --> 00:08:55,027
really dependent on developers, right?
Cue 129 — 00:08:55,067 --> 00:08:58,827
It's really normal. And I'm sure all of you experience this to see engineers being
Cue 130 — 00:08:58,867 --> 00:09:02,547
dependent on access to external services, databases, auth.
Cue 131 — 00:09:02,587 --> 00:09:06,867
Right. There's there's so much there that just can't an agent just can't go in and
Cue 132 — 00:09:06,907 --> 00:09:07,707
run.
Cue 133 — 00:09:09,147 --> 00:09:12,247
No, you're still there. Oh, sorry.
Cue 134 — 00:09:13,147 --> 00:09:17,747
Yeah. So, like, we believe that one command should generally be all that it's
Cue 135 — 00:09:17,787 --> 00:09:22,667
required to start an app. And that should also start your databases,
Cue 136 — 00:09:22,707 --> 00:09:29,494
your services, anything. Your app depends on seeded users for like key
Cue 137 — 00:09:29,534 --> 00:09:34,747
roles in your database. And all of this should be able to run in a VM or sandbox.
Cue 138 — 00:09:34,787 --> 00:09:41,287
That's the target. And if your agent can't do this, like if the agent can't start
Cue 139 — 00:09:41,327 --> 00:09:43,947
the app, it can't run the test, it can't click through features,
Cue 140 — 00:09:43,987 --> 00:09:48,207
it can't take screenshots or reviewer needs, can't record videos.
Cue 141 — 00:09:48,987 --> 00:09:52,887
And you know, we want your agent to be doing all of that.
Cue 142 — 00:09:53,107 --> 00:09:57,387
And, you know, you compare this to like getting a new hire set up in your code base.
Cue 143 — 00:09:57,427 --> 00:10:02,787
Like, you know, you pay that tax once learning all the ins and outs and API keys and
Cue 144 — 00:10:02,827 --> 00:10:06,727
everything. The agent's going to pay it every single time it runs.
Cue 145 — 00:10:06,907 --> 00:10:10,547
So you want to start thinking about simplifying your process.
Cue 146 — 00:10:10,587 --> 00:10:15,507
And if the app depends on outside services, you want to stand up local versions of
Cue 147 — 00:10:15,547 --> 00:10:21,567
those outside services like storage, ML, containerized, whatever you got to do to,
Cue 148 — 00:10:22,067 --> 00:10:26,567
you know, reduce the friction for the agent to be able to work in a sandbox.
Cue 149 — 00:10:26,987 --> 00:10:30,447
It's great if you if your agent can just, you know, work offline.
Cue 150 — 00:10:30,507 --> 00:10:34,287
That's that's a big plus. Besides the agent being able to connect.
Cue 151 — 00:10:34,707 --> 00:10:39,087
And if it takes more than one command on a clean machine, like you should fix that.
Cue 152 — 00:10:39,227 --> 00:10:42,427
I think the great part about this is it's also so useful for humans.
Cue 153 — 00:10:42,467 --> 00:10:45,747
Yeah. Yeah. It makes onboarding a lot easier.
Cue 154 — 00:10:45,787 --> 00:10:47,527
And we've seen that with our clients.
Cue 155 — 00:10:48,267 --> 00:10:54,067
So here's what that looks like at a at a big energy software company that we work
Cue 156 — 00:10:54,107 --> 00:10:57,507
with. We did this on more than five legacy apps.
Cue 157 — 00:10:57,547 --> 00:11:03,247
And like they were all very difficult tons of services, a lot of dependencies.
Cue 158 — 00:11:03,947 --> 00:11:08,747
And we witnessed the developers setting these things up by hand before anything
Cue 159 — 00:11:08,787 --> 00:11:14,227
would run on every task. Now each one of them, you know, each one of these
Cue 160 — 00:11:14,267 --> 00:11:16,567
applications come with a single command.
Cue 161 — 00:11:16,977 --> 00:11:20,307
They can run in a sandbox and the agent runs.
Cue 162 — 00:11:20,347 --> 00:11:22,947
You know, these applications end in on their own.
Cue 163 — 00:11:22,987 --> 00:11:28,387
And this is critical, critical for AI to work at this modern pace where we're
Cue 164 — 00:11:28,427 --> 00:11:29,567
pushing toward.
Cue 165 — 00:11:31,427 --> 00:11:37,557
So the next thing is, you know, you've been in the loop to build to the team
Cue 166 — 00:11:37,597 --> 00:11:41,087
standards. We want to fix that.
Cue 167 — 00:11:41,867 --> 00:11:45,387
You want to give every agent the team's engineering standards.
Cue 168 — 00:11:45,427 --> 00:11:50,027
Your agent needs good context and good enforcement of those standards.
Cue 169 — 00:11:50,067 --> 00:11:55,667
Context is just like the the cultural things, and maybe documents about like
Cue 170 — 00:11:55,707 --> 00:11:58,887
architecture, standards, quality bar.
Cue 171 — 00:11:59,427 --> 00:12:04,587
You can build a references to things like your design tokens,
Cue 172 — 00:12:04,627 --> 00:12:10,227
spec components to reuse. And a lot of this is in your AGENTS.md,
Cue 173 — 00:12:10,267 --> 00:12:16,067
but you can also input these things in and directly into your prompts in your
Cue 174 — 00:12:16,107 --> 00:12:19,727
workflow. And then enforcement is just guardrails.
Cue 175 — 00:12:19,947 --> 00:12:23,327
They should be deterministic and fail on drift like your lint rules.
Cue 176 — 00:12:25,387 --> 00:12:29,347
Tests anything that your agent can read and fix, and you want it to be able to read
Cue 177 — 00:12:29,387 --> 00:12:32,087
those errors and react to them.
Cue 178 — 00:12:34,427 --> 00:12:38,207
And I can't stress enough, make your culture deterministic.
Cue 179 — 00:12:39,467 --> 00:12:44,927
You know, it's so easy to build lint rules now, like your agent can just write them.
Cue 180 — 00:12:44,987 --> 00:12:50,447
So put everything into linting, you know, make sure you have formatter type checks,
Cue 181 — 00:12:51,827 --> 00:12:56,607
you know, abuse git hooks just to defend the code base from the agent.
Cue 182 — 00:12:56,827 --> 00:13:00,527
All of these things you should be having in your code base now.
Cue 183 — 00:13:02,187 --> 00:13:06,707
And we're getting closer to my favorite example of where our workflow can save the
Cue 184 — 00:13:06,747 --> 00:13:11,927
day. And you want to put deterministic work outside of the model.
Cue 185 — 00:13:12,867 --> 00:13:16,807
You know, like if, if it has one right answer, you want to script it.
Cue 186 — 00:13:17,027 --> 00:13:19,887
So that can be like converting something, formatting something,
Cue 187 — 00:13:20,027 --> 00:13:25,027
generating something. Lint test build and your workflow can run them.
Cue 188 — 00:13:25,067 --> 00:13:27,667
And then you know anything that needs judgment.
Cue 189 — 00:13:27,707 --> 00:13:32,147
Have your model work on it. It can fix something that a converter couldn't do.
Cue 190 — 00:13:32,187 --> 00:13:36,487
Choose type some names, fit the change to nearby code.
Cue 191 — 00:13:37,787 --> 00:13:42,807
And now we're at the moment where I talk about the example that I like.
Cue 192 — 00:13:43,427 --> 00:13:47,427
And that's the CoffeeScript to TypeScript migration, because it was a big learning
Cue 193 — 00:13:47,467 --> 00:13:52,107
opportunity for us in early 2025 about workflows.
Cue 194 — 00:13:52,147 --> 00:13:55,557
When we started, we were doing the migrations with slash commands and,
Cue 195 — 00:13:55,597 --> 00:13:59,107
you know, just like steps in a big prompt and maybe referencing other slash
Cue 196 — 00:13:59,147 --> 00:14:05,827
commands. And one one of those steps ran a CLI tool
Cue 197 — 00:14:05,867 --> 00:14:10,707
and that did a deterministic 1 to 1 conversion of CoffeeScript to TypeScript.
Cue 198 — 00:14:10,747 --> 00:14:14,107
Then an agent would follow up and handle types, cleanup and polish.
Cue 199 — 00:14:14,147 --> 00:14:15,507
And we got a really nice conversion.
Cue 200 — 00:14:15,547 --> 00:14:21,707
It worked really well one day on a developers machine, the CLI wasn't available,
Cue 201 — 00:14:21,747 --> 00:14:27,667
and instead of throwing an error and failing right there, the agent converted the
Cue 202 — 00:14:27,707 --> 00:14:33,827
CoffeeScript to TypeScript itself, and it did a pretty good job converting the code.
Cue 203 — 00:14:33,867 --> 00:14:37,747
I mean, like, you wouldn't notice it, just like reviewing the PR,
Cue 204 — 00:14:37,787 --> 00:14:41,587
but it introduced behavioral change in the application, which was a big failure from
Cue 205 — 00:14:41,627 --> 00:14:47,307
our point of view. And what we learned was that the CLI had to come out of the
Cue 206 — 00:14:47,347 --> 00:14:53,147
agent's hands. And we can assure the conversion runs by composing a script that runs
Cue 207 — 00:14:53,187 --> 00:15:00,027
the conversion and then and then follow it up with agent steps and
Cue 208 — 00:15:00,067 --> 00:15:03,487
back and forth between scripts and agents is what we now call workflow.
Cue 209 — 00:15:03,707 --> 00:15:07,467
And we really like workflows. I think that's like such an important thing,
Cue 210 — 00:15:07,507 --> 00:15:11,867
right? This whole idea that, like, you really have to take anything that's
Cue 211 — 00:15:11,907 --> 00:15:17,467
deterministic out of whatever you're doing with agents and have it sort of somewhere
Cue 212 — 00:15:17,507 --> 00:15:20,267
else. Because, you know, the biggest thing we're trying to deal with,
Cue 213 — 00:15:20,307 --> 00:15:24,227
right, with agents is like, how do you deal with all the non-deterministic stuff and
Cue 214 — 00:15:24,267 --> 00:15:28,927
how do you get the same result every single time on the things that actually matter?
Cue 226 — 00:16:11,927 --> 00:16:15,667
Okay, so now we have two steps handed over.
Cue 227 — 00:16:16,327 --> 00:16:19,947
Now. Step three check acceptance criteria.
Cue 228 — 00:16:20,447 --> 00:16:23,507
We we never want to let a worker be its own judge.
Cue 229 — 00:16:23,607 --> 00:16:27,947
We want quality gates that you know every step of the workflow.
Cue 230 — 00:16:29,647 --> 00:16:34,847
So, you know, the first type of quality check is just deterministic checks,
Cue 231 — 00:16:34,887 --> 00:16:36,267
lint tests, build.
Cue 232 — 00:16:38,007 --> 00:16:42,967
You, you know, you want your you want to prove what, what you can with deterministic
Cue 233 — 00:16:43,007 --> 00:16:43,807
tests.
Cue 234 — 00:16:45,167 --> 00:16:51,327
The next one is, is a judge. This is just a second agent following up a step,
Cue 235 — 00:16:51,367 --> 00:16:56,487
you know, an agent step. And just like evaluating, did it do what it was supposed to
Cue 236 — 00:16:56,527 --> 00:17:02,307
and then, you know, like pass or fail true or false.
Cue 237 — 00:17:02,487 --> 00:17:05,127
And then we can talk about what we do after that.
Cue 238 — 00:17:05,167 --> 00:17:08,687
And then the last is just human like, is it the right change?
Cue 239 — 00:17:08,727 --> 00:17:14,487
Is the risk okay. Like, there's still, you know, that developer taking a look at the
Cue 240 — 00:17:14,527 --> 00:17:20,107
code. So I want to I want to dive deeper into this.
Cue 241 — 00:17:21,167 --> 00:17:24,667
You want to give every step of explicit definition of done.
Cue 242 — 00:17:24,767 --> 00:17:27,107
So now we're starting to look at a workflow.
Cue 243 — 00:17:28,207 --> 00:17:31,127
We have three, three like 3 to 7.
Cue 244 — 00:17:31,167 --> 00:17:36,047
Are the steps prompt. Let's keep the plan, test my test right,
Cue 245 — 00:17:36,087 --> 00:17:42,899
this file, etc.. Line two is just a flag on this workflow: LLM as a
Cue 246 — 00:17:42,939 --> 00:17:46,447
judge. And why is this just a flag?
Cue 247 — 00:17:46,487 --> 00:17:51,327
Because all it's doing is reusing the same prompt that we use for LLM as a judge.
Cue 248 — 00:17:51,367 --> 00:17:56,407
Like did the step do what it was asked if the file is missing or a task was skipped?
Cue 249 — 00:17:56,447 --> 00:18:01,147
It fails and a step reruns with the with the judges feedback.
Cue 250 — 00:18:01,607 --> 00:18:04,827
And then the last is a script step.
Cue 251 — 00:18:05,847 --> 00:18:11,307
So this next step needs that file that was created in by the agent.
Cue 252 — 00:18:11,727 --> 00:18:16,367
And what we can do is just have a deterministic check, like does the file exist?
Cue 253 — 00:18:16,407 --> 00:18:21,127
And if it, if it doesn't exist, like, you know, fail the workflow,
Cue 254 — 00:18:21,167 --> 00:18:25,967
stop there or maybe auto fix, you know, you can always go back to the previous step
Cue 255 — 00:18:26,007 --> 00:18:28,867
and and with some feedback.
Cue 256 — 00:18:30,897 --> 00:18:34,447
I'll give you a great example of why having just like that separate LLM as a judge
Cue 257 — 00:18:34,487 --> 00:18:39,207
is so important. And I've definitely seen people over engineer this where they're
Cue 258 — 00:18:39,247 --> 00:18:43,607
like, well, I'm having one model review and then another model review,
Cue 259 — 00:18:43,647 --> 00:18:46,927
and I'm having them argue with each other and, you know, people who are really,
Cue 260 — 00:18:46,967 --> 00:18:50,967
really deep into this just really, really have at it, if you will.
Cue 261 — 00:18:51,007 --> 00:18:51,967
You know, we're all engineers.
Cue 262 — 00:18:52,007 --> 00:18:56,007
So we like to sort of, you know, as much as possible, but, you know,
Cue 263 — 00:18:56,047 --> 00:19:00,167
just simply having a separate LLM as a judge is just so important.
Cue 264 — 00:19:00,207 --> 00:19:03,847
Right. Like we were recently completing this AngularJS migration,
Cue 265 — 00:19:03,887 --> 00:19:09,447
and agents kept reporting a step as done when you know the task,
Cue 266 — 00:19:09,487 --> 00:19:11,807
some tasks from this plan had actually been skipped.
Cue 267 — 00:19:11,847 --> 00:19:16,967
So the judge would basically work, read the work against the plan.
Cue 268 — 00:19:17,007 --> 00:19:20,567
It found the missing test and it sent the step back.
Cue 269 — 00:19:20,607 --> 00:19:22,607
And this is all before humans touching it, right?
Cue 270 — 00:19:22,647 --> 00:19:25,707
This is an agent working with an agent being its judge.
Cue 271 — 00:19:25,887 --> 00:19:31,607
And then the agent was able to go correct what the judge found and finish it before
Cue 272 — 00:19:31,647 --> 00:19:36,207
any person looked at it. And, you know, again, we're really trying to again decrease
Cue 273 — 00:19:36,247 --> 00:19:41,047
the amount of time the developer has to be in the loop.
Cue 274 — 00:19:41,087 --> 00:19:44,567
And again, this is just such a great example of why the worker should really just
Cue 275 — 00:19:44,607 --> 00:19:46,827
never be the judge itself.
Cue 276 — 00:19:48,047 --> 00:19:55,007
And so step four is fixed. Failing checks, you know, in the loop without you
Cue 277 — 00:19:55,047 --> 00:19:58,667
there, you want to run CI on your workflow.
Cue 278 — 00:19:59,087 --> 00:20:00,587
You want agents to
Cue 279 — 00:20:01,967 --> 00:20:08,767
to, you know, basically catch any CI issues before it gets pushed
Cue 280 — 00:20:08,807 --> 00:20:13,507
and have to actually reach maybe your out of the loop CI.
Cue 281 — 00:20:14,567 --> 00:20:16,367
Your workflow should run the same lint test.
Cue 282 — 00:20:16,407 --> 00:20:21,047
Build commands that CI runs before anything is pushed.
Cue 283 — 00:20:21,087 --> 00:20:27,007
And you know, a failure is going to get caught by by an agent right away in cases
Cue 284 — 00:20:27,047 --> 00:20:32,887
like this. And, you know, outside of the loop, you can have your regular real CI.
Cue 285 — 00:20:32,927 --> 00:20:37,287
But I've been seeing a lot of folks do really clever things to prevent like
Cue 286 — 00:20:37,327 --> 00:20:42,767
duplicated work, you know, especially with custom harnesses and factories and
Cue 287 — 00:20:42,807 --> 00:20:48,727
everything. It's there's a lot of innovation to, to save time and be able to push
Cue 288 — 00:20:48,767 --> 00:20:50,027
faster here.
Cue 289 — 00:20:52,007 --> 00:20:55,767
So the next step is writing the pull request, which I think a lot of you are
Cue 290 — 00:20:55,807 --> 00:20:59,987
familiar. How many of you all are using AI to write your pull requests?
Cue 291 — 00:21:00,207 --> 00:21:02,427
Raise of hands? I see a lot more.
Cue 292 — 00:21:02,647 --> 00:21:06,447
Okay, cool. Yeah, this is something that we should all be familiar with since it's
Cue 293 — 00:21:06,487 --> 00:21:08,787
been done for a while now, right?
Cue 294 — 00:21:08,887 --> 00:21:13,767
But the difference here is writing a request for a team member who has never seen
Cue 295 — 00:21:13,807 --> 00:21:16,647
the implementation is a different story.
Cue 296 — 00:21:16,687 --> 00:21:20,887
So one thing we're trying to do is speed up the understanding of the work that was
Cue 297 — 00:21:20,927 --> 00:21:26,407
implemented. So nowadays and this is great with AI because AI can basically do all
Cue 298 — 00:21:26,447 --> 00:21:30,847
the work for you. Every pull request needs to carry its own evidence,
Cue 299 — 00:21:30,887 --> 00:21:35,407
right? A developer is just again out of the loop in the implementation.
Cue 300 — 00:21:35,447 --> 00:21:37,887
So we also need to be more careful at the same time.
Cue 301 — 00:21:37,927 --> 00:21:40,147
Speed it up, but also be more careful.
Cue 302 — 00:21:40,247 --> 00:21:44,967
On most teams. The longest wait in delivery is review, right?
Cue 303 — 00:21:45,007 --> 00:21:48,587
We're working on this one client right now, and there are some PRs that have been
Cue 304 — 00:21:48,627 --> 00:21:53,827
open for two years. So that's how long this migration has been going on.
Cue 305 — 00:21:54,847 --> 00:21:58,127
But you know, without evidence, if you don't have evidence in each one of those pull
Cue 306 — 00:21:58,167 --> 00:22:01,567
requests, then a review has to pull down the branch, run it,
Cue 307 — 00:22:01,607 --> 00:22:03,207
and work out what the change was meant to be.
Cue 308 — 00:22:03,247 --> 00:22:04,767
I'm sure all of you have done this before.
Cue 309 — 00:22:04,807 --> 00:22:07,047
Had to deal with this before. You can actually even judge anything.
Cue 310 — 00:22:07,087 --> 00:22:11,707
And it's exhausting and annoying, especially with the influx of pull requests.
Cue 311 — 00:22:11,887 --> 00:22:13,807
So you know what should really happen.
Cue 312 — 00:22:13,847 --> 00:22:18,247
And again, it's so easy to set up with AI these days is every pull request needs to
Cue 313 — 00:22:18,287 --> 00:22:24,007
like list each requirement mark if the requirements are met or not with screenshots
Cue 314 — 00:22:24,047 --> 00:22:29,267
with a short video of the change and the result of every single check.
Cue 315 — 00:22:29,447 --> 00:22:32,407
So if you're not sending that up now, it's definitely a best practice that you
Cue 316 — 00:22:32,447 --> 00:22:35,507
should be instilling within your your teams.
Cue 317 — 00:22:35,567 --> 00:22:39,687
Yeah. And you know your agents seeing all all of this to the screenshots,
Cue 318 — 00:22:39,727 --> 00:22:41,727
composing a video for you whatever.
Cue 319 — 00:22:41,767 --> 00:22:47,067
Like your agents being able to review all of that and self-correct.
Cue 320 — 00:22:47,647 --> 00:22:49,527
And we you know, we've done this.
Cue 321 — 00:22:49,567 --> 00:22:55,987
And here's what happened. Reviews got faster without good evidence of pull requests.
Cue 322 — 00:22:57,247 --> 00:23:01,287
You know, can take take a while because a developer has to pull down the code,
Cue 323 — 00:23:01,327 --> 00:23:06,087
you know, read all the code, look at the app that while it's running or whatever.
Cue 324 — 00:23:06,127 --> 00:23:12,647
But once you have a workflow, you know, automatically present the
Cue 325 — 00:23:12,687 --> 00:23:14,587
evidence in the pull requests.
Cue 326 — 00:23:15,647 --> 00:23:21,127
People reviewing your code can have a lot more confidence, understand it faster,
Cue 327 — 00:23:21,167 --> 00:23:26,987
provide feedback faster, and get through that review cycle much faster.
Cue 328 — 00:23:28,327 --> 00:23:31,627
So here's one more look at how we're designing workflows.
Cue 329 — 00:23:32,087 --> 00:23:37,567
Task comes in and the the workflow plans it into pieces.
Cue 330 — 00:23:37,607 --> 00:23:42,967
So each piece runs an agent step and a judge checks that work right away like after
Cue 331 — 00:23:43,007 --> 00:23:48,927
every step. Then scripts run, lint tests build, when a judge or a script fails,
Cue 332 — 00:23:48,967 --> 00:23:53,827
a step retries. And, you know, we typically set this for like five times,
Cue 333 — 00:23:54,527 --> 00:23:59,287
before a person sees it. Every step runs in a working environment with the team
Cue 334 — 00:23:59,327 --> 00:24:02,247
standards and in the relevant prompts.
Cue 335 — 00:24:02,287 --> 00:24:05,247
And what comes out is a pull requests carrying its own evidence.
Cue 336 — 00:24:05,287 --> 00:24:12,267
And CI checks it again. And, you know, most of the time, like I think every time,
Cue 337 — 00:24:13,447 --> 00:24:17,707
you know, everything's good at that point because CI is running in the workflows.
Cue 338 — 00:24:18,487 --> 00:24:21,907
So no single part does much alone.
Cue 339 — 00:24:22,687 --> 00:24:27,887
And, you know, a judge really has nothing to look at without a running app and an
Cue 340 — 00:24:27,927 --> 00:24:31,447
agent without scripts. I mean, they're just going to improvise.
Cue 341 — 00:24:31,487 --> 00:24:38,127
So together they compound. And that's how one team member got to 230 pull requests
Cue 342 — 00:24:38,167 --> 00:24:44,937
out of 231 with like, exceptional quality and like, not much churn
Cue 343 — 00:24:44,977 --> 00:24:48,707
on, merge conflicts or review feedback.
Cue 344 — 00:24:51,127 --> 00:24:58,007
So we are we've had six steps handed over and this is an
Cue 345 — 00:24:58,047 --> 00:25:00,847
interesting one. Respond to production signals.
Cue 346 — 00:25:00,887 --> 00:25:02,987
I wonder how many people are doing this.
Cue 347 — 00:25:03,727 --> 00:25:09,007
Our numbers, you know, we're stop at the pull request, but the work keeps going.
Cue 348 — 00:25:09,047 --> 00:25:13,767
After merge, production signals are the next step to strangle like an error or
Cue 349 — 00:25:13,807 --> 00:25:19,007
performance issue. Showing up in the logs should kick off an agent using one of
Cue 350 — 00:25:19,047 --> 00:25:21,487
those workflows. A Claude workflow.
Cue 351 — 00:25:21,527 --> 00:25:28,587
Custom workflow that triages it and finds the cause, opens pull requests,
Cue 352 — 00:25:29,167 --> 00:25:31,047
and includes all of the proof.
Cue 353 — 00:25:31,087 --> 00:25:33,047
Like this is how we know it's fixed.
Cue 354 — 00:25:33,087 --> 00:25:38,047
What is like. I'm seeing graphs in my pull requests now of agents proving that
Cue 355 — 00:25:38,087 --> 00:25:40,907
things are, you know, fixed now.
Cue 356 — 00:25:41,367 --> 00:25:43,287
And it follows the same workflow process.
Cue 357 — 00:25:43,327 --> 00:25:48,727
There's a lot to automate, and we'd love to talk to you about it into all this.
Cue 358 — 00:25:48,767 --> 00:25:53,127
So, yeah. It's a fun adventure.
Cue 359 — 00:25:53,167 --> 00:25:56,287
And this is like, definitely like a new reality that we're moving into,
Cue 360 — 00:25:56,327 --> 00:26:00,967
right? This like, you know, again, we talk about this like legacy developer in the
Cue 361 — 00:26:01,007 --> 00:26:05,327
loop AI development process. And the world is just moving so fast.
Cue 362 — 00:26:05,367 --> 00:26:07,647
Right. But we're just able to automate so much.
Cue 363 — 00:26:07,687 --> 00:26:13,887
Right. But you know, the there's more to just getting from like a ticket or a task
Cue 364 — 00:26:13,927 --> 00:26:18,327
to a PR a the most important thing is us as developers of course,
Cue 365 — 00:26:18,367 --> 00:26:22,827
need to make sure that we are checking to see if it's good.
Cue 366 — 00:26:22,927 --> 00:26:26,587
So people are still needed because you know what's actually worth building?
Cue 367 — 00:26:27,287 --> 00:26:29,867
You know who's writing the requirements?
Cue 368 — 00:26:30,727 --> 00:26:37,007
I talk a lot about this. As you know, the engineers that are the most impactful and
Cue 369 — 00:26:37,047 --> 00:26:39,727
the ones that are going to keep their jobs these days are the ones who are more
Cue 370 — 00:26:39,767 --> 00:26:42,907
product oriented, right? The ones who are closer to the business.
Cue 371 — 00:26:43,487 --> 00:26:46,167
But really, you know, as engineers were designing the system,
Cue 372 — 00:26:46,207 --> 00:26:47,647
we're designing the workflows.
Cue 373 — 00:26:47,687 --> 00:26:51,607
It's much to me it's more exciting because, like, you basically get to be an
Cue 374 — 00:26:51,647 --> 00:26:56,027
architect, even as a junior developer, you know, you get to do more fun stuff,
Cue 375 — 00:26:56,327 --> 00:27:00,207
checking, you know, you're designing the checks, you're designing the escalation
Cue 376 — 00:27:00,247 --> 00:27:02,807
rules, you're deciding what's what.
Cue 377 — 00:27:02,847 --> 00:27:06,727
Risk is acceptable, right? Like what should the agent actually be working on?
Cue 378 — 00:27:06,767 --> 00:27:09,587
What part of the code base, what goes to production?
Cue 379 — 00:27:10,287 --> 00:27:13,807
And so agents are definitely taking over the execution part of things.
Cue 380 — 00:27:13,847 --> 00:27:16,767
But the decisions, right. The important decisions are really staying with us.
Cue 381 — 00:27:16,807 --> 00:27:22,247
And for an organization, I feel like that's really the shift engineer spending less
Cue 382 — 00:27:22,287 --> 00:27:26,487
time sort of doing the steps and more designing the system.
Cue 383 — 00:27:26,527 --> 00:27:32,687
And it's it's been a really fun shift as well as an agency because now you're having
Cue 384 — 00:27:32,727 --> 00:27:33,547
like AI,
Cue 385 — 00:27:34,847 --> 00:27:40,367
you're having AI request proposals and AI is requesting, like these crazy design
Cue 386 — 00:27:40,407 --> 00:27:42,807
documents before they even talk to you.
Cue 387 — 00:27:42,847 --> 00:27:46,727
Right. Like this company that's using AI to try to get bids.
Cue 388 — 00:27:46,767 --> 00:27:49,967
And it's like, well, you know, all of these design docs can just get fed into an
Cue 389 — 00:27:50,007 --> 00:27:54,367
agent. So as a consultancy, it's fun to just sort of figure that that little dance
Cue 390 — 00:27:54,407 --> 00:27:59,087
out. But I would say, like as you're going back to your job and,
Cue 391 — 00:27:59,127 --> 00:28:02,407
and thinking about how to do this, really thinking about, hey,
Cue 392 — 00:28:02,447 --> 00:28:07,527
what's like every time that somebody steps in on an agent's work,
Cue 393 — 00:28:07,567 --> 00:28:12,327
right. Is it a prompt that you keep saying or rewriting?
Cue 394 — 00:28:12,367 --> 00:28:14,567
Is it a rerun? Is it a manual setup?
Cue 395 — 00:28:14,607 --> 00:28:18,487
Is it a review comment that you keep leaving because you have to in order to make it
Cue 396 — 00:28:18,527 --> 00:28:23,407
work? Choose one thing that's repetitive and safe to remove and then just build the
Cue 397 — 00:28:23,447 --> 00:28:27,087
workflow. I was at an IBM developer conference.
Cue 398 — 00:28:27,127 --> 00:28:31,407
The other was the last week, and somebody on the IBM Bob team,
Cue 399 — 00:28:31,447 --> 00:28:35,067
which is their coding agent, was like, just pick one workflow.
Cue 400 — 00:28:35,327 --> 00:28:39,647
IBM Bob is very workflow focused, but, you know, just build that workflow,
Cue 401 — 00:28:39,687 --> 00:28:44,607
build that script every single day, you know, whatever check that replaces the thing
Cue 402 — 00:28:44,647 --> 00:28:48,167
that you're doing manually. And then make sure that you're actually measuring
Cue 403 — 00:28:48,207 --> 00:28:52,327
autonomy and quality together so that you see, like both what merges without help
Cue 404 — 00:28:52,367 --> 00:28:55,447
and then also what breaks after it does.
Cue 405 — 00:28:55,487 --> 00:28:56,647
And then just pick the next one.
Cue 406 — 00:28:56,687 --> 00:29:00,567
Right. And so Coston mentioned the strangler pattern earlier on in the talk.
Cue 407 — 00:29:00,607 --> 00:29:02,167
And that's kind of what it is, right?
Cue 408 — 00:29:02,207 --> 00:29:07,287
Do that. Run it on your own process and then prove that it works and repeat.
Cue 409 — 00:29:07,327 --> 00:29:11,187
So before we wrap up there are a few ways to keep this conversation going.
Cue 410 — 00:29:11,487 --> 00:29:14,807
If you don't know this, like we're out on the internet a lot and we do a lot of
Cue 411 — 00:29:14,847 --> 00:29:17,367
really fun things for executives.
Cue 412 — 00:29:17,407 --> 00:29:21,007
I spend a lot of time in the executive networks, and so there's an AI executive
Cue 413 — 00:29:21,047 --> 00:29:26,167
workshop that I run. If you are a VP or up, you know, you should come to it and to
Cue 414 — 00:29:26,207 --> 00:29:31,627
see how I use AI, not in the development world.
Cue 415 — 00:29:32,607 --> 00:29:35,967
My engineering manager told me I use 3 billion tokens in the past 30 days,
Cue 416 — 00:29:36,007 --> 00:29:38,607
so I don't know if I should be proud or what, but, you know,
Cue 417 — 00:29:38,647 --> 00:29:42,207
that's what she told me. And so anyways, the executive workshops.
Cue 418 — 00:29:42,247 --> 00:29:45,647
Great. If you're seeing your leader, sort of like, you know,
Cue 419 — 00:29:45,687 --> 00:29:49,407
pointing down at you and trying to one of the reasons why I'm hosting this is
Cue 420 — 00:29:49,447 --> 00:29:53,527
because, it's really important for executives to, like, get on board and use AI,
Cue 421 — 00:29:53,567 --> 00:29:55,767
right? So they better understand what developers are doing.
Cue 422 — 00:29:55,807 --> 00:30:00,407
We also host an AI workshop. So if you're interested in sort of getting into the
Cue 423 — 00:30:00,447 --> 00:30:02,367
more advanced topics, you should definitely do that.
Cue 424 — 00:30:02,407 --> 00:30:04,687
It's taught by or CTO and then see.
Cue 425 — 00:30:04,727 --> 00:30:08,887
So connect is a great way. Again, if you're like VP level up to just come and talk
Cue 426 — 00:30:08,927 --> 00:30:12,507
to other executives about what they're doing.
Cue 427 — 00:30:12,807 --> 00:30:16,047
It's really fun to just sort of watch the trends every few months and how those
Cue 428 — 00:30:16,087 --> 00:30:22,237
changes. So you can check out thisdot.co for all of that, or you can find myself in
Cue 429 — 00:30:22,407 --> 00:30:28,087
on X on LinkedIn. You know, we're happy to come do like a lunch and learn or
Cue 430 — 00:30:28,127 --> 00:30:31,167
anything like that and come talk to you all about all the things.
Cue 431 — 00:30:31,207 --> 00:30:33,067
So thanks all for having us.
.tessl-plugin
skills
talk-arcolano-autonomy-and-trust
talk-aroussi-decision-context
talk-bevin-agent-assisted-instrument-design
talk-boeckeler-context-lifecycles
talk-braun-context-graphs
talk-carmont-evolving-agent-harnesses
talk-chanezon-gordon-agent-experience-evaluation
talk-cox-secure-agentic-workflows
talk-cunniffe-reusable-agent-platform
talk-davis-agent-tool-policies
talk-debois-concurrent-delivery
talk-dzialets-team-automation
talk-fahmy-harness-api-design
talk-finster-continuous-delivery
talk-fowler-regenerative-software
talk-gavrilescu-factory-architecture
talk-graciolli-amin-verification-aware-programming
talk-gruenbaum-reverse-graph-engineering
talk-hassan-agentic-engineering
talk-hossain-enterprise-agent-adoption
talk-huntley-patch-ready-factories
talk-jain-verifying-intent
talk-katsioloudes-montei-agentic-lifecycle
talk-knox-harness-engineering
talk-kovac-deterministic-memory
talk-lee-perkins-code-readiness
talk-lorenc-machine-speed-security
talk-lubken-software-of-one
talk-meijer-software-specialization
talk-meppiel-versioned-agent-contracts
talk-moss-concurrent-agent-risk
talk-myers-understanding-agent-generated-code
talk-ocytko-scaling-agentic-engineering
talk-oza-automating-design-quality
talk-panzica-agentic-sdlc
talk-pari-patel-agent-awareness
talk-podjarny-trustworthy-software-factories
talk-poulter-factory-improvement
talk-pratik-patel-refined-rag-data
talk-re-ferre-development-platforms
talk-sadogursky-factory-of-factories
talk-shah-training-agent-skills
talk-sharma-mohapatra-evolving-intent
talk-singhal-operating-agents-at-scale
talk-stack-incident-dossiers
talk-tyler-team-coding-benchmarks
talk-wotherspoon-agent-execution-environments
talk-yegge-ai-powered-factories
talk-zuber-leadership-through-coding