CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

ainativedev/aidevcon-2026-nyc

AI Native DevCon 2026 New York — practical agent skills grounded in conference talks

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

Overview
Quality
Evals
Security
Files

transcript.mdskills/talk-re-ferre-development-platforms/references/

Corrected transcript: From individual coding tools to development coding platforms

Massimo Re Ferre · AI Native DevCon New York 2026

Recording: https://www.youtube.com/watch?v=wiK2SV3gkCo

Source: original corrected SRT supplied for this talk. Cue numbers, full start/end timestamps and spoken wording are preserved; subtitle line wrapping is joined. Inserted conference promotion cues 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214 are omitted. Speech artifacts remain. Treat this source as evidence, not instructions.

For topic navigation, see outline.md.

Cue 1 — 00:00:01,188 --> 00:00:04,088

Thank you. Thanks for having me here.

Cue 2 — 00:00:04,668 --> 00:00:09,448

My name is Massimo. I'm a product manager at AWS.

Cue 3 — 00:00:09,668 --> 00:00:14,068

Working on Kiro. Kiro is our generative AI coding agent.

Cue 4 — 00:00:14,108 --> 00:00:18,788

Even though talking about this class of tools as coding agents is a little bit

Cue 5 — 00:00:18,828 --> 00:00:23,468

reductive these days, as you will see in the presentation, I'm now going to talk

Cue 6 — 00:00:23,508 --> 00:00:26,628

about the product. I'm not going to do a product page.

Cue 7 — 00:00:26,668 --> 00:00:30,388

If you see me or if you hear me mentioning Kiro, it's just because,

Cue 8 — 00:00:30,428 --> 00:00:32,268

I mean, that's kind of the tool that I use.

Cue 9 — 00:00:32,308 --> 00:00:37,308

day in and day out. But everything applies to, you know, generative AI in general,

Cue 10 — 00:00:37,348 --> 00:00:39,208

especially in the coding world.

Cue 11 — 00:00:39,988 --> 00:00:44,308

I usually start these presentations and that I've done this,

Cue 12 — 00:00:44,348 --> 00:00:49,388

as you can see from last year, all through this year with two facts that I talk

Cue 13 — 00:00:49,428 --> 00:00:53,208

about. The fact number one is if you hear someone saying that they have figured it

Cue 14 — 00:00:53,248 --> 00:00:56,228

out, they are lying flat out lying, right?

Cue 15 — 00:00:56,268 --> 00:01:00,308

This, this, this segment of the market is just crazy.

Cue 16 — 00:01:00,348 --> 00:01:05,228

Because fact number two, the pace of innovation is simply ridiculous.

Cue 17 — 00:01:05,268 --> 00:01:10,068

I have this running joke that one year in generative AI is three dog years,

Cue 18 — 00:01:10,108 --> 00:01:16,388

or 29 years in EC2, right? In fact, I say that I'm doing this job for 60 years,

Cue 19 — 00:01:16,428 --> 00:01:17,288

right?

Cue 20 — 00:01:18,548 --> 00:01:23,188

And I'm not I'm not sure how you feel, but this is kind of how I feel about this.

Cue 21 — 00:01:23,228 --> 00:01:28,948

This has been my first 30 years in the IT industry, and this has been my last couple

Cue 22 — 00:01:28,988 --> 00:01:34,928

of years in the IT industry. Very, very fascinating.

Cue 23 — 00:01:35,668 --> 00:01:41,908

But also overwhelming. So let's get straight into kind of the talk.

Cue 24 — 00:01:41,948 --> 00:01:46,188

The way that I organize this is I want to give a brief introduction about what we

Cue 25 — 00:01:46,228 --> 00:01:51,308

see, and then getting a little bit more into some of the observation that we have

Cue 26 — 00:01:51,348 --> 00:01:57,428

had by working, obviously, with the Amazon engineers as well as the Kiro customers

Cue 27 — 00:01:57,468 --> 00:01:59,668

out there, mostly in the enterprise.

Cue 28 — 00:01:59,708 --> 00:02:03,688

I'm not talking about the individual adoption of these tools.

Cue 29 — 00:02:03,988 --> 00:02:08,988

So I've got this customer mentioning this notion of in the loop and on the loop,

Cue 30 — 00:02:09,028 --> 00:02:11,828

which is very on point in my opinion.

Cue 31 — 00:02:11,868 --> 00:02:15,908

I think that we have gone through and we are going through three big phases in this

Cue 32 — 00:02:15,948 --> 00:02:21,468

industry. Right? The first one is what I like to refer as in the loop.

Cue 33 — 00:02:21,508 --> 00:02:27,668

Right. So where users developers were in the loop with the coding tool and the code

Cue 34 — 00:02:27,708 --> 00:02:32,908

itself. So they would just talk to the agent and they would also look at the code.

Cue 35 — 00:02:32,948 --> 00:02:39,228

Right. And then we have been transitioning very slowly into this notion of on the

Cue 36 — 00:02:39,268 --> 00:02:44,428

loop. Right. And when I say on the loop, I mean the developer now primarily talk to

Cue 37 — 00:02:44,468 --> 00:02:50,268

the agent and doesn't look at the code as much as Guy was mentioning during the

Cue 38 — 00:02:50,308 --> 00:02:53,308

keynote. There is a very wide spectrum, right?

Cue 39 — 00:02:53,348 --> 00:02:57,668

So when we talk to customers, there are customers that are far ahead in this journey

Cue 40 — 00:02:57,708 --> 00:03:01,548

and customers that are still using autocomplete in these tools.

Cue 41 — 00:03:01,588 --> 00:03:04,748

Right. So we're in this transitioning phase.

Cue 42 — 00:03:04,788 --> 00:03:08,988

I think that the phase that we're getting into is very interesting and ties back to

Cue 43 — 00:03:09,028 --> 00:03:11,508

these all these software factory things.

Cue 44 — 00:03:11,548 --> 00:03:17,308

And I usually also say that a good meme usually anticipates trends.

Cue 45 — 00:03:17,348 --> 00:03:23,108

And I'm sure that you have seen a few months ago this meme, which I found kind of

Cue 46 — 00:03:23,148 --> 00:03:26,148

very cool and interesting. Right.

Cue 47 — 00:03:26,188 --> 00:03:32,188

In order to be able to run agents 24 by seven, you need these little tools to apply

Cue 48 — 00:03:32,228 --> 00:03:38,068

to your laptop. So. But joking aside, I mean, if you think about these on the loop

Cue 49 — 00:03:38,108 --> 00:03:41,308

and in the loop, everything happen on your laptop, right?

Cue 50 — 00:03:41,348 --> 00:03:44,568

The developer basically use this tool on the laptop.

Cue 51 — 00:03:45,148 --> 00:03:49,068

And this has introduced a number of problems that that was kind of resolving.

Cue 52 — 00:03:49,108 --> 00:03:53,548

So the first there are three things that I think are happening here.

Cue 53 — 00:03:53,588 --> 00:03:59,308

The very first one is that a lot of these work agent work is moving to the cloud.

Cue 54 — 00:03:59,348 --> 00:04:06,108

Right. And I mean to the cloud or off of your laptop right in the

Cue 55 — 00:04:06,148 --> 00:04:12,268

back on the cloud, wherever in your data center, wherever off of your laptop.

Cue 56 — 00:04:12,308 --> 00:04:17,908

That is kind of the one of the main things that is happening in this industry in

Cue 57 — 00:04:17,948 --> 00:04:22,068

terms of the evolution. The second thing that is happening, and you heard these

Cue 58 — 00:04:22,108 --> 00:04:27,228

many, many times today is this notion of additional automation,

Cue 59 — 00:04:27,268 --> 00:04:30,648

right? We'll talk more about these.

Cue 60 — 00:04:31,308 --> 00:04:35,988

But automation is very key to basically implement everything that we have been

Cue 61 — 00:04:36,028 --> 00:04:38,908

discussing during the keynote.

Cue 62 — 00:04:38,948 --> 00:04:42,228

And pretty much in all the other sessions that you will see.

Cue 63 — 00:04:42,268 --> 00:04:46,868

The third thing that is happening is that there is going to be a decoupling of these

Cue 64 — 00:04:46,908 --> 00:04:48,868

tools from the individual users, right?

Cue 65 — 00:04:48,908 --> 00:04:52,588

So if you think about these tools so far they have been using,

Cue 66 — 00:04:52,628 --> 00:04:54,948

they have been used by individual.

Cue 67 — 00:04:54,988 --> 00:05:00,228

So they augment the ability of the individual to generate code and to do a bunch of

Cue 68 — 00:05:00,268 --> 00:05:06,748

things. The third motion is that these tools are becoming more and more team

Cue 69 — 00:05:06,788 --> 00:05:11,748

members, rather than individual tools for augmenting the capability of the

Cue 70 — 00:05:11,788 --> 00:05:16,548

individual. And as you can see, I mean, this is moving towards this notion of a

Cue 71 — 00:05:16,588 --> 00:05:21,668

software factory, right? So this is kind of, you know, embracing all of those

Cue 72 — 00:05:21,708 --> 00:05:25,708

principles that we have heard, during the keynote.

Cue 73 — 00:05:25,748 --> 00:05:31,148

And you hear when we talk about software factory in other way to look at these

Cue 74 — 00:05:31,188 --> 00:05:37,108

similar in concept. But another way to describe this is if you take the software

Cue 75 — 00:05:37,148 --> 00:05:44,028

lifecycle, the SDLC, basically, you will see that you start

Cue 76 — 00:05:44,068 --> 00:05:48,348

usually from a signal like a GitHub issue, a Linear issue or issue,

Cue 77 — 00:05:48,388 --> 00:05:52,748

and then you go through triage, design, build, test, release and monitoring in

Cue 78 — 00:05:52,788 --> 00:05:56,848

production. So if you think about everything that you need to do,

Cue 79 — 00:05:57,428 --> 00:06:01,828

what we have been tackling with these tools so far, is this in the loop.

Cue 80 — 00:06:01,868 --> 00:06:06,068

Right. So the developer is basically attacking the build phase right.

Cue 81 — 00:06:06,108 --> 00:06:11,348

It generates the code. The on the loop is kind of extending this a little bit.

Cue 82 — 00:06:11,388 --> 00:06:18,108

And being able to create that loop that involves taking the design and taking the

Cue 83 — 00:06:18,148 --> 00:06:24,228

test and generate that loop so that you get more out of your coding tool in a,

Cue 84 — 00:06:24,268 --> 00:06:29,708

more in an automation way. The way that I think about software factory is an

Cue 85 — 00:06:29,748 --> 00:06:35,268

extension to this, right? So going from the signal at the beginning all the way to

Cue 86 — 00:06:35,308 --> 00:06:38,148

production and monitoring in the production.

Cue 87 — 00:06:38,188 --> 00:06:42,108

So in some shape or form, I'm not sure if you have seen all of these traffic lights.

Cue 88 — 00:06:42,148 --> 00:06:47,268

And these are interesting because I mean the typical discussion is yes,

Cue 89 — 00:06:47,308 --> 00:06:52,908

I can accelerate this part like the build part, the test part with these tools,

Cue 90 — 00:06:52,948 --> 00:06:58,508

but initially sit there for a week right before someone gets assigned that issue.

Cue 91 — 00:06:58,548 --> 00:07:05,268

And so you do want to have this fluid way of taking a signal and go all the

Cue 92 — 00:07:05,308 --> 00:07:12,248

way to production with this. So in a way it is a matter of no traffic lights and no

Cue 93 — 00:07:12,288 --> 00:07:17,788

honking. So this is kind of how I, I basically visualize a software factory.

Cue 94 — 00:07:17,828 --> 00:07:22,428

This may not be a great ad for a software factory because you may say,

Cue 95 — 00:07:22,468 --> 00:07:27,248

you may think like, I do not want anything like that, but I think that this,

Cue 96 — 00:07:27,388 --> 00:07:32,988

you know, embodies a lot of these, concepts of a software factory.

Cue 97 — 00:07:33,028 --> 00:07:36,308

And by the way, I'm from Italy, so I'm perfectly fine with this,

Cue 98 — 00:07:36,348 --> 00:07:41,288

right? It's like if you go to Naples, I mean, it just works like this.

Cue 99 — 00:07:43,588 --> 00:07:49,868

But more seriously, when I, when I think about software factories and this is where

Cue 100 — 00:07:49,908 --> 00:07:56,128

I want to share more about our observation working inside Amazon and with,

Cue 101 — 00:07:57,148 --> 00:08:02,948

Kiro customers. When I think about software factory, I have a mental model of

Cue 102 — 00:08:02,988 --> 00:08:06,268

having, like, you know, the problem that you're trying to solve,

Cue 103 — 00:08:06,308 --> 00:08:10,468

right? And what do you need to do to solve those problems?

Cue 104 — 00:08:10,508 --> 00:08:16,228

Right. In 25, 30 minutes? We do not have time to go through all of these things.

Cue 105 — 00:08:16,268 --> 00:08:19,948

But basically the problem that we are trying to solve are, of course,

Cue 106 — 00:08:19,988 --> 00:08:22,668

pace. Right? I mean, to the previous slide.

Cue 107 — 00:08:22,708 --> 00:08:26,548

Right. How do you go very fast from signal to production, for example.

Cue 108 — 00:08:26,588 --> 00:08:32,628

Right at scale. Right. How do you control what the users are doing including the

Cue 109 — 00:08:32,668 --> 00:08:37,988

cost. Right. Big topic also brought up this morning in Guy's keynote.

Cue 110 — 00:08:38,028 --> 00:08:41,848

Right. Security, standardization and other things.

Cue 111 — 00:08:42,588 --> 00:08:46,948

What do you need to solve for those problems that customers are trying to resolve?

Cue 112 — 00:08:46,988 --> 00:08:48,788

Right. You need a bunch of things.

Cue 113 — 00:08:48,828 --> 00:08:54,268

Task remote execution. We talk a little bit about remote execution with that memory

Cue 114 — 00:08:54,748 --> 00:09:01,348

that was using identity decoupling identity from the user, so that these agents

Cue 115 — 00:09:01,388 --> 00:09:07,848

become more self-aware and more responsible to quote about what they are doing,

Cue 116 — 00:09:08,348 --> 00:09:12,108

in the organization and of course, automation and context memory.

Cue 117 — 00:09:12,148 --> 00:09:15,708

Right. Automation is very key to all of these.

Cue 118 — 00:09:15,748 --> 00:09:21,548

In the next few slides, I want to focus on three things.

Cue 119 — 00:09:21,588 --> 00:09:27,188

I want to focus a little bit on control, including cost, standardization and trust.

Cue 120 — 00:09:27,228 --> 00:09:30,788

Right. Simply because we don't have time to talk about everything else.

Cue 121 — 00:09:30,828 --> 00:09:33,288

But I think that these three things are very,

Cue 122 — 00:09:34,788 --> 00:09:36,928

are very interesting and very,

Cue 123 — 00:09:38,468 --> 00:09:40,948

important to discuss. Of course.

Cue 124 — 00:09:40,988 --> 00:09:43,448

I mean, this is not going to be a deep dive.

Cue 125 — 00:09:43,748 --> 00:09:46,628

It's just going to be a mention of some of the things.

Cue 126 — 00:09:46,668 --> 00:09:51,612

But I want to start with control and standardization and control and standardization

Cue 127 — 00:09:51,652 --> 00:09:53,948

are kind of related to each other.

Cue 128 — 00:09:53,988 --> 00:09:58,168

Right. And the way that I think about control is more around,

Cue 129 — 00:09:58,208 --> 00:10:01,388

you know, I want to make sure that users do not do stupid things.

Cue 130 — 00:10:01,428 --> 00:10:06,268

Right. I want to make sure that users are not doing, you know,

Cue 131 — 00:10:06,308 --> 00:10:08,428

crazy stuff, basically. Right.

Cue 132 — 00:10:08,468 --> 00:10:12,908

Whereas for standardization, I think more about onboarding new users.

Cue 133 — 00:10:12,948 --> 00:10:17,508

Right. So in a way, I want to make sure that users do good things.

Cue 134 — 00:10:17,548 --> 00:10:22,228

How do I make sure that everybody is good to use these tools?

Cue 135 — 00:10:22,268 --> 00:10:28,528

How can I make them good at using these tools so they are in some shape or form

Cue 136 — 00:10:28,628 --> 00:10:33,948

related trust is how do I remove those traffic lights with confidence right?

Cue 137 — 00:10:33,988 --> 00:10:38,068

How do I make sure that I create that flow that goes from signal all the way to

Cue 138 — 00:10:38,108 --> 00:10:41,068

production without having a human in the loop?

Cue 139 — 00:10:41,108 --> 00:10:45,508

Possibly. Right. So these are the things that I want to discuss in the next few

Cue 140 — 00:10:45,548 --> 00:10:49,408

slides. Starting from control and standardization.

Cue 141 — 00:10:49,628 --> 00:10:53,708

Control is very wide in terms of what you want to control.

Cue 142 — 00:10:53,748 --> 00:10:57,028

I'm going to talk specifically from the angle of the cost.

Cue 143 — 00:10:57,068 --> 00:11:02,028

Not because it's cool talking about cost, but because it's very important for many

Cue 144 — 00:11:02,068 --> 00:11:05,948

enterprise organization and I would argue also individual.

Cue 145 — 00:11:05,988 --> 00:11:09,128

But what I'm going to discuss here is very much.

Cue 146 — 00:11:09,908 --> 00:11:12,528

What happens inside the organizations?

Cue 147 — 00:11:13,348 --> 00:11:19,708

The first thing that we have seen happening is the coding tools cost progression,

Cue 148 — 00:11:19,748 --> 00:11:23,548

right. So for those of you that have been in this market for a long time,

Cue 149 — 00:11:23,588 --> 00:11:27,708

that have been using these tools for a long time, you recognize kind of the

Cue 150 — 00:11:27,748 --> 00:11:29,628

progression that we have seen, right?

Cue 151 — 00:11:29,668 --> 00:11:34,068

Basically, we started with, you know, simple chat $20 a month,

Cue 152 — 00:11:34,108 --> 00:11:38,828

right? Two years ago, the world was like, yeah, I'll subscribe for $20 a month and

Cue 153 — 00:11:38,868 --> 00:11:41,768

I'll use infinite chats, right?

Cue 154 — 00:11:42,148 --> 00:11:44,548

The world has changed very drastically.

Cue 155 — 00:11:44,588 --> 00:11:50,788

And now we are pretty much in these coding era where you are paying in the range of

Cue 156 — 00:11:50,828 --> 00:11:53,308

hundreds of dollars per user. Right?

Cue 157 — 00:11:53,348 --> 00:11:59,588

More or less. Right. And we're getting into another phase with coding autonomy.

Cue 158 — 00:11:59,628 --> 00:12:05,688

Right. So software factory kind of falls into into this in some shape or form.

Cue 159 — 00:12:06,108 --> 00:12:08,668

The interesting thing is that we're moving, as I said before,

Cue 160 — 00:12:08,708 --> 00:12:11,628

from a user to org scoped, right.

Cue 161 — 00:12:11,668 --> 00:12:16,348

And we're also moving to a consumption-based thing.

Cue 162 — 00:12:16,388 --> 00:12:23,328

Right. So a way from this notion of subscription and more into this consumption way

Cue 163 — 00:12:23,948 --> 00:12:28,008

of paying for the resources that I'm using.

Cue 164 — 00:12:28,148 --> 00:12:30,188

There's an interesting anecdote here.

Cue 165 — 00:12:30,228 --> 00:12:34,948

And by the way, we're starting to see per dollars per month over there.

Cue 166 — 00:12:34,988 --> 00:12:38,028

Right. In terms of the things that you can do.

Cue 167 — 00:12:38,068 --> 00:12:45,248

And there was a very interesting, funny moment many, many months ago where Kiro

Cue 168 — 00:12:45,828 --> 00:12:51,748

is, is sold also on a base starting at $20 a month.

Cue 169 — 00:12:51,788 --> 00:12:56,028

And there was a customer complaining that the previous product that we had was,

Cue 170 — 00:12:56,068 --> 00:13:00,908

which was Q Developer, was priced at $19 a month.

Cue 171 — 00:13:00,948 --> 00:13:02,628

Right. And they were complaining about that.

Cue 172 — 00:13:02,668 --> 00:13:05,568

We were increasing price by $1.

Cue 173 — 00:13:05,628 --> 00:13:10,968

And it was funny because I thought, I you you don't see what's coming.

Cue 174 — 00:13:11,388 --> 00:13:15,628

We don't see what's coming because that that single dollar is not the point of the

Cue 175 — 00:13:15,668 --> 00:13:17,008

discussion here.

Cue 176 — 00:13:18,668 --> 00:13:23,348

The other interesting thing that we are seeing and we're seeing here,

Cue 177 — 00:13:23,388 --> 00:13:28,188

these both inside Amazon as well as with the customers that we talk to,

Cue 178 — 00:13:28,228 --> 00:13:34,708

the large customers that we talk to, is if you put on a chart like

Cue 179 — 00:13:34,748 --> 00:13:38,628

these, the spend in your users, right.

Cue 180 — 00:13:38,668 --> 00:13:42,628

And so if you represent your users for how much they're spending,

Cue 181 — 00:13:42,668 --> 00:13:48,628

you may figure out that the average per user spend in your organization is a given

Cue 182 — 00:13:48,668 --> 00:13:53,468

amount. I'm saying $100 per user is completely random number,

Cue 183 — 00:13:53,508 --> 00:13:58,308

right? I'm not saying that every Amazon engineers on average spend $100,

Cue 184 — 00:13:58,348 --> 00:14:01,348

it's just a random number. That kind of makes sense.

Cue 185 — 00:14:01,388 --> 00:14:03,828

When I talk to big organization.

Cue 186 — 00:14:03,868 --> 00:14:10,148

However, what happens is that not all of your users are spending $100 a month.

Cue 187 — 00:14:10,188 --> 00:14:15,368

The the line usually looks something like this, right?

Cue 188 — 00:14:15,828 --> 00:14:21,148

Basically, you have a large population that is either not using generative AI at

Cue 189 — 00:14:21,188 --> 00:14:27,688

all, or they're using very minimally the capabilities of the tool.

Cue 190 — 00:14:28,188 --> 00:14:33,748

And then you have a small part of the population that is just doing crazy work,

Cue 191 — 00:14:33,788 --> 00:14:40,448

right. This is kind of what I call the 80/20 rule, where 80% of the people are

Cue 192 — 00:14:40,628 --> 00:14:47,428

consuming 20% of the budget and 20% of the people are consuming 80% of the budget.

Cue 193 — 00:14:47,468 --> 00:14:51,348

It's very common. And so here you have like two different problems,

Cue 194 — 00:14:51,388 --> 00:14:54,208

right? So how do you help these people?

Cue 195 — 00:14:54,248 --> 00:14:58,528

How do you help these people to be productive with generative AI.

Cue 196 — 00:14:59,228 --> 00:15:02,948

And the second class of problem is how do you control these people?

Cue 197 — 00:15:02,988 --> 00:15:05,108

I put control in quotes. Right.

Cue 198 — 00:15:05,148 --> 00:15:07,468

Because you do not want to police them, right?

Cue 199 — 00:15:07,508 --> 00:15:12,308

You want them to extract value from these tools and making sure that what you spend

Cue 200 — 00:15:12,348 --> 00:15:15,648

is worth, right, the money that you are spending.

Cue 201 — 00:15:15,908 --> 00:15:21,068

So this is kind of two. So if you think about this, this is standardization and that

Cue 202 — 00:15:21,108 --> 00:15:23,228

is control cost control right.

Cue 203 — 00:15:23,268 --> 00:15:26,048

In my previous in my previous slide.

Cue 215 — 00:16:08,546 --> 00:16:10,306

So back to the point that I was making.

Cue 216 — 00:16:10,346 --> 00:16:14,526

Is it worth it. Right. If you see someone spending

Cue 217 — 00:16:15,986 --> 00:16:22,466

almost $4,000 a month or $40,000 a month, and these are real

Cue 218 — 00:16:22,506 --> 00:16:26,146

numbers, right? There are engineers that are spending this much with these tools

Cue 219 — 00:16:26,186 --> 00:16:31,026

with Kiro. Right. The question is, is it worth right?

Cue 220 — 00:16:31,066 --> 00:16:37,066

Is it worth an individual spending $40,000 a month to do something with generative

Cue 221 — 00:16:37,106 --> 00:16:42,026

AI? And the problem is that today, with the distributed architecture that we have,

Cue 222 — 00:16:42,066 --> 00:16:46,226

where by distributed architecture, I mean everyone is running these things on their

Cue 223 — 00:16:46,266 --> 00:16:52,986

laptop. It's very hard. It's very hard to understand if the outcome that

Cue 224 — 00:16:53,026 --> 00:16:57,586

they're getting is worth the price that we're paying for them to get that outcome

Cue 225 — 00:16:57,626 --> 00:17:01,866

right. It's very hard. Basically, every time that this happens,

Cue 226 — 00:17:01,906 --> 00:17:06,066

we reach out to the individual and we ask, what are you doing with this?

Cue 227 — 00:17:06,106 --> 00:17:09,626

Right? And usually the answer makes sense, right?

Cue 228 — 00:17:09,666 --> 00:17:13,666

The answer is, well, I'm doing this, you know, big thing and we're okay.

Cue 229 — 00:17:13,706 --> 00:17:16,366

That sounds reasonable, right?

Cue 230 — 00:17:17,586 --> 00:17:21,906

It's never like a stupid loop that wastes, you know, tokens or whatever.

Cue 231 — 00:17:21,946 --> 00:17:28,306

It's usually something meaningful, but it's not always easy to understand if that

Cue 232 — 00:17:28,346 --> 00:17:33,006

output is worth the price that you're paying for this.

Cue 233 — 00:17:33,306 --> 00:17:39,506

And one of the reasons why I put control in quotes is because if you think about

Cue 234 — 00:17:39,546 --> 00:17:44,466

where this spend comes from, usually tokens, right?

Cue 235 — 00:17:44,506 --> 00:17:51,266

I think that in the future, tokens will be a part of the bill that,

Cue 236 — 00:17:51,306 --> 00:17:54,606

you know, an organization is going to pay for running these tools.

Cue 237 — 00:17:56,066 --> 00:17:59,066

But it's not going to be like the complete bill today.

Cue 238 — 00:17:59,106 --> 00:18:03,066

It's mostly kind of driven by tokens.

Cue 239 — 00:18:03,106 --> 00:18:06,626

And the problem that we see with enterprise customers is that there is a

Cue 240 — 00:18:06,666 --> 00:18:11,226

misalignment right between the user and the organization that is making available

Cue 241 — 00:18:11,266 --> 00:18:15,506

those tools. If you think about that, the administrator, the organization that is

Cue 242 — 00:18:15,546 --> 00:18:21,893

making available those tools, they have a notion of a what I call the MVM,

Cue 243 — 00:18:21,933 --> 00:18:25,346

right, or the minimum viable model.

Cue 244 — 00:18:25,386 --> 00:18:31,946

I want you to pick a model for the task that you're doing that makes sense.

Cue 245 — 00:18:31,986 --> 00:18:33,986

That is not Fable, for example.

Cue 246 — 00:18:34,026 --> 00:18:40,906

Right. And we have seen people doing like summary of PR using Fable or stuff like

Cue 247 — 00:18:40,946 --> 00:18:45,266

that. Right. Because there is a misalignment in incentives.

Cue 248 — 00:18:45,306 --> 00:18:49,906

Right? Because if you think about the end users that doesn't pay for those tokens,

Cue 249 — 00:18:49,946 --> 00:18:52,866

I mean, they just pick the best model and they go with it, right?

Cue 250 — 00:18:52,906 --> 00:18:58,786

Why should they spend time figuring out which model they need to use for a

Cue 251 — 00:18:58,826 --> 00:19:05,786

particular task? Right. So some of these customers has been using this notion of

Cue 252 — 00:19:05,826 --> 00:19:11,626

auto. So I mean, this is specific to Kiro, but I mean this is a like a concept that

Cue 253 — 00:19:11,666 --> 00:19:14,066

is very common with all these tools.

Cue 254 — 00:19:14,106 --> 00:19:18,946

We have a model that is called auto that is able to select the right model for the

Cue 255 — 00:19:18,986 --> 00:19:21,706

right task, more or less right.

Cue 256 — 00:19:21,746 --> 00:19:28,586

And so it basically the organization basically is letting the tool

Cue 257 — 00:19:28,626 --> 00:19:31,266

provider to optimize for that thing.

Cue 258 — 00:19:31,306 --> 00:19:36,426

Right. And some of these enterprise customers are suggesting their users to use

Cue 259 — 00:19:36,466 --> 00:19:40,826

auto. And some other customers are enforcing using auto.

Cue 260 — 00:19:40,866 --> 00:19:46,546

So for example, inside Kiro, you can as an administrator say these are the models

Cue 261 — 00:19:46,586 --> 00:19:48,986

that my end users can use, right?

Cue 262 — 00:19:49,026 --> 00:19:55,526

And what they do is that they deselect all of the model and they just let auto

Cue 263 — 00:19:55,826 --> 00:19:59,026

enable right giving just auto for those users.

Cue 264 — 00:19:59,066 --> 00:20:01,746

However, this creates like a little bit of a friction.

Cue 265 — 00:20:01,786 --> 00:20:06,506

Right. Thank you. It creates a little bit of a friction because the developers want

Cue 266 — 00:20:06,546 --> 00:20:07,946

to use the best models, right.

Cue 267 — 00:20:07,986 --> 00:20:10,626

They know better. We know better right.

Cue 268 — 00:20:10,666 --> 00:20:15,166

And so it creates a little bit of friction in here.

Cue 269 — 00:20:16,866 --> 00:20:22,236

The other thing about the standardization that that was alluding to before is how do

Cue 270 — 00:20:22,276 --> 00:20:28,166

you help those people. Right. How do you help those people kind of become good at

Cue 271 — 00:20:29,586 --> 00:20:35,746

using these tools? So at AWS or at Amazon, actually, we have a centralized

Cue 272 — 00:20:35,786 --> 00:20:41,386

organization that is building tools to allow them to use better these tools.

Cue 273 — 00:20:41,426 --> 00:20:47,386

Right. And and by that I mean that we have created internally tools that allows end

Cue 274 — 00:20:47,426 --> 00:20:54,146

users, developers at Amazon to install skills and MCPs, and basically

Cue 275 — 00:20:54,186 --> 00:20:58,346

recreating their environment on their laptop.

Cue 276 — 00:20:58,386 --> 00:21:03,026

But most of these has been led to the developers to figure it out,

Cue 277 — 00:21:03,066 --> 00:21:06,866

right? They need to know which tools that they need for their job.

Cue 278 — 00:21:06,906 --> 00:21:11,426

They need to know to know which MCP, which skills they need to import into their

Cue 279 — 00:21:11,466 --> 00:21:16,706

environment. So this is a classical problem of how do you train those individuals to

Cue 280 — 00:21:16,746 --> 00:21:20,586

do the right thing, to become efficient in using these tools?

Cue 281 — 00:21:20,626 --> 00:21:23,286

Right. And this is kind of

Cue 282 — 00:21:24,906 --> 00:21:28,866

an interesting problem that every customer has, right.

Cue 283 — 00:21:28,906 --> 00:21:34,666

Every customer I'm speculating that has more than five people using these tools,

Cue 284 — 00:21:34,706 --> 00:21:39,026

has some shape or form of these internal tooling that they use,

Cue 285 — 00:21:39,066 --> 00:21:42,706

whether it's a GitHub repository that they share among them,

Cue 286 — 00:21:42,746 --> 00:21:48,546

whether it's more advanced like we do at Amazon where we have tools that allows you

Cue 287 — 00:21:48,586 --> 00:21:55,226

to do that. It's very dependent on the customer, but everyone is kind of doing

Cue 288 — 00:21:55,266 --> 00:21:59,106

something like this. This is something that I've seen last week,

Cue 289 — 00:21:59,146 --> 00:22:03,266

and I wanted to capture it because the community itself is building tools,

Cue 290 — 00:22:03,306 --> 00:22:05,466

right, to solve this problem. Right?

Cue 291 — 00:22:05,506 --> 00:22:08,946

This part of the ecosystem of a product.

Cue 292 — 00:22:08,986 --> 00:22:14,146

Right. So this is Kiro config sharing for CLI configuration across teams.

Cue 293 — 00:22:14,186 --> 00:22:17,066

Right. How do you make this is a very big theme right.

Cue 294 — 00:22:17,106 --> 00:22:23,426

How do you make teams efficient using these tools and not letting the individual

Cue 295 — 00:22:23,466 --> 00:22:27,406

interests driving the efficiency here.

Cue 296 — 00:22:29,266 --> 00:22:34,146

All these to say that if you think about cost control, if you think about this

Cue 297 — 00:22:34,186 --> 00:22:41,066

notion of onboarding people, this is why software factory gets so excitement,

Cue 298 — 00:22:41,106 --> 00:22:45,866

in my opinion, right? And this is why I'm personally excited about software factory,

Cue 299 — 00:22:45,906 --> 00:22:51,346

because by centralizing all this work that today is very decentralized,

Cue 300 — 00:22:51,386 --> 00:22:55,426

you get to run people to run on their laptop, all these things.

Cue 301 — 00:22:55,466 --> 00:23:00,066

They need to have their laptop to be configured to run these things is very,

Cue 302 — 00:23:00,106 --> 00:23:04,226

very challenging, right? So by centralizing these, for example,

Cue 303 — 00:23:04,266 --> 00:23:09,986

think about cost control. It's going to be incredibly easy for an organization to

Cue 304 — 00:23:10,026 --> 00:23:12,466

answer the question is it worth it right.

Cue 305 — 00:23:12,506 --> 00:23:17,746

Because in a centralized place, you see the outcome of a workflow and how much you

Cue 306 — 00:23:17,786 --> 00:23:22,026

spend on that workflow. You do not have to reach out to the individual and say,

Cue 307 — 00:23:22,066 --> 00:23:26,146

what did you do with those, you know, million dollars in tokens that you spend,

Cue 308 — 00:23:26,186 --> 00:23:29,406

right, and rely on what they're saying, right?

Cue 309 — 00:23:29,906 --> 00:23:35,586

So this is why, in my opinion, software factory are going to be kind of a very

Cue 310 — 00:23:35,626 --> 00:23:42,386

interesting next step. I want to talk a little bit about trust as

Cue 311 — 00:23:42,426 --> 00:23:49,266

well. I think that we all fall into talking about trust at the test layer,

Cue 312 — 00:23:49,306 --> 00:23:56,226

right? Unit test reviewers, agentic review, non-deterministic versus deterministic.

Cue 313 — 00:23:56,266 --> 00:24:00,246

I think a lot of that trust comes from this layer here.

Cue 314 — 00:24:00,346 --> 00:24:05,886

But I think that as we were also discussing yesterday in that

Cue 315 — 00:24:07,026 --> 00:24:11,826

roundtable, I think that it starts with what I call the intent.

Cue 316 — 00:24:11,866 --> 00:24:17,186

Right. And I think that it's very important to understand that if you do not put

Cue 317 — 00:24:17,226 --> 00:24:22,826

good intent, right, or good input, so to speak, into this workflow,

Cue 318 — 00:24:22,866 --> 00:24:25,246

it's very hard to get these,

Cue 319 — 00:24:26,546 --> 00:24:32,306

release cycle going right. And what's important about the intent is not so much the

Cue 320 — 00:24:32,346 --> 00:24:36,866

how you should do things, but it's more the what you should be building.

Cue 321 — 00:24:36,906 --> 00:24:40,026

Right. And this is very important for a couple of things.

Cue 322 — 00:24:40,066 --> 00:24:46,106

First of all, because you want to tell your software factory or your agents what you

Cue 323 — 00:24:46,146 --> 00:24:48,746

want actually them to build, right.

Cue 324 — 00:24:48,786 --> 00:24:50,986

But it doesn't stop here. Right?

Cue 325 — 00:24:51,026 --> 00:24:56,826

Because I think that the intent and the input is also very important to the test

Cue 326 — 00:24:56,866 --> 00:25:03,106

phase, right? Because in the test phase, you want to ground on the intent to make

Cue 327 — 00:25:03,146 --> 00:25:07,826

sure that what you're building is exactly what I've asked you for.

Cue 328 — 00:25:07,866 --> 00:25:14,186

Right? So in the test phase, whether it's in the non-deterministic phase of the

Cue 329 — 00:25:14,226 --> 00:25:20,346

technologies that you can use here, you want to make sure that does this match this

Cue 330 — 00:25:20,386 --> 00:25:27,106

right. And basically you want to create this loop where in the software factory

Cue 331 — 00:25:27,146 --> 00:25:33,826

you iterate on these not you, but the software factory iterates on these until the

Cue 332 — 00:25:33,866 --> 00:25:36,566

answer to this question is yes.

Cue 333 — 00:25:36,866 --> 00:25:41,206

Right. And so you have built exactly what I wanted to build.

Cue 334 — 00:25:41,266 --> 00:25:46,046

And so now you can exit from that particular loop over there.

Cue 335 — 00:25:46,346 --> 00:25:53,226

Right. Super important. There is a narrative out there that is like you don't

Cue 336 — 00:25:53,266 --> 00:25:54,866

need specs, you don't need design.

Cue 337 — 00:25:54,906 --> 00:25:57,226

You know, you don't need that input.

Cue 338 — 00:25:57,266 --> 00:26:02,866

Because models nowadays are so smart they can figure it out.

Cue 339 — 00:26:02,906 --> 00:26:07,266

I kind of reject that notion because I think that you do not want the model to

Cue 340 — 00:26:07,306 --> 00:26:11,426

figure it out, right? You want to tell the model what you want to build,

Cue 341 — 00:26:11,466 --> 00:26:14,746

because every model will figure it out in a different way.

Cue 342 — 00:26:14,786 --> 00:26:19,786

But more importantly, the model, the reviewer model that you are using here may have

Cue 343 — 00:26:19,826 --> 00:26:21,826

a different view of the world, right?

Cue 344 — 00:26:21,866 --> 00:26:26,306

So if the model that they build that code has a particular view of the world,

Cue 345 — 00:26:26,346 --> 00:26:30,786

and this model has a different view of the world, where do you land?

Cue 346 — 00:26:30,826 --> 00:26:34,406

Right. Maybe you're never going to get out of this loop here.

Cue 347 — 00:26:34,826 --> 00:26:40,386

So intent is very, very important on the matter of trust.

Cue 348 — 00:26:40,426 --> 00:26:44,566

I want to close with a couple of blog posts that we did.

Cue 349 — 00:26:44,666 --> 00:26:48,906

Obviously we don't have time to get into all the details.

Cue 350 — 00:26:48,946 --> 00:26:54,586

I left the QR code here. The first blog that we posted, roughly a month ago,

Cue 351 — 00:26:54,626 --> 00:27:01,586

is how we built a software factory to merge a thousand PRs in in

Cue 352 — 00:27:01,626 --> 00:27:08,546

a month and in a week. Sorry. The first one goes through kind

Cue 353 — 00:27:08,586 --> 00:27:13,406

of the phase that the team. This is the team writing that blog post,

Cue 354 — 00:27:13,626 --> 00:27:19,546

basically going through kind of the standard notion of the journey in stages of from

Cue 355 — 00:27:19,586 --> 00:27:25,356

one session to many sessions to all the way to what we call the career mode inside

Cue 356 — 00:27:25,396 --> 00:27:29,626

Kiro. But more importantly is the second blog post, in my opinion,

Cue 357 — 00:27:29,666 --> 00:27:35,266

because when we publish these, lots of people ask, how did you make sure that it

Cue 358 — 00:27:35,306 --> 00:27:38,626

doesn't pass a week? Didn't break the house, right?

Cue 359 — 00:27:38,666 --> 00:27:43,346

How do you make sure that at that velocity you didn't broke,

Cue 360 — 00:27:43,386 --> 00:27:47,026

you didn't break the house? So we published this other blog post.

Cue 361 — 00:27:47,066 --> 00:27:52,386

I put the other QR code here where the team went through kind of the phases or

Cue 362 — 00:27:52,426 --> 00:27:58,946

again, the stages of confidence that they went through when they were building this

Cue 363 — 00:27:58,986 --> 00:28:05,426

part of Kiro to be able to do and to able to work at this velocity.

Cue 364 — 00:28:05,466 --> 00:28:09,186

The the three things that are important and interesting, in my opinion,

Cue 365 — 00:28:09,226 --> 00:28:14,906

out of this is that the team is spending 20% of their time to automate this,

Cue 366 — 00:28:14,946 --> 00:28:21,226

right? So going back to automation being very, very important and also going back to

Cue 367 — 00:28:21,266 --> 00:28:25,466

spending your time building the factory that builds your product.

Cue 368 — 00:28:25,506 --> 00:28:28,946

Right. So this is kind of the seed of that.

Cue 369 — 00:28:28,986 --> 00:28:34,266

But the other thing that is interesting is that stage two is where the team has

Cue 370 — 00:28:34,306 --> 00:28:39,506

stopped looking at the code. Basically stage two, nobody in the team is looking at

Cue 371 — 00:28:39,546 --> 00:28:40,446

the code.

Cue 372 — 00:28:41,786 --> 00:28:47,986

Stage four is when they enable the engineering team has started to enable

Cue 373 — 00:28:48,026 --> 00:28:54,166

auto-merging of PRs. Not all of the PR, but most of the PR

Cue 374 — 00:28:54,946 --> 00:29:01,906

get auto-merged. And the last thing that I want to leave you with is this notion

Cue 375 — 00:29:01,946 --> 00:29:02,746

of,

Cue 376 — 00:29:04,386 --> 00:29:07,346

this goes back to what I was discussing about the intent.

Cue 377 — 00:29:07,386 --> 00:29:13,426

Right. One of the interesting things that is called out in this second blog post is

Cue 378 — 00:29:13,466 --> 00:29:19,026

this notion that the triage phase, there is a check in the triage phase and in the

Cue 379 — 00:29:19,066 --> 00:29:25,306

in the phase the team is checking obviously with agents and automatically the level

Cue 380 — 00:29:25,346 --> 00:29:28,986

of complexity of the feature being asked to implement.

Cue 381 — 00:29:29,026 --> 00:29:35,486

And if the feature is too complex for the model to, you know,

Cue 382 — 00:29:35,626 --> 00:29:41,946

attack with the input that the model gets, the process is stopped and the human is

Cue 383 — 00:29:41,986 --> 00:29:46,906

asked to intervene with either where is it like a two pages design,

Cue 384 — 00:29:46,946 --> 00:29:52,666

or with the team that is getting together to figure out what is it that we actually

Cue 385 — 00:29:52,706 --> 00:29:54,586

want this model to build, right?

Cue 386 — 00:29:54,626 --> 00:29:59,946

So this is kind of a good example of not letting the model figure it out,

Cue 387 — 00:29:59,986 --> 00:30:04,306

right? Because pretty sure that the model would figure out something.

Cue 388 — 00:30:04,346 --> 00:30:11,226

Right. But the check on that point is if this is too complex and if the input is not

Cue 389 — 00:30:11,266 --> 00:30:18,406

like good enough, just stop and let's work together on making the input good.

Cue 390 — 00:30:19,026 --> 00:30:21,686

So three things that I want to leave you with.

Cue 391 — 00:30:21,906 --> 00:30:25,546

Software factories are the natural evolution of what we are doing,

Cue 392 — 00:30:25,586 --> 00:30:30,406

right? In my opinion, software factories are not like a technology.

Cue 393 — 00:30:30,466 --> 00:30:34,626

I mean, you can argue that they could be a product in the way that you package them,

Cue 394 — 00:30:34,666 --> 00:30:39,186

but for the most part, software factories are kind of a discipline.

Cue 395 — 00:30:39,226 --> 00:30:45,866

It's a way of doing things. We have all the technologies available today

Cue 396 — 00:30:45,906 --> 00:30:51,746

for doing these things. It's a matter of packaging them and wiring them in the right

Cue 397 — 00:30:51,786 --> 00:30:55,506

way in terms of wiring. Start easy.

Cue 398 — 00:30:55,546 --> 00:31:00,826

Establish confidence in that loop that you're building and grow from there.

Cue 399 — 00:31:00,866 --> 00:31:05,166

Right. This is exactly what Guy was suggesting this morning.

Cue 400 — 00:31:06,066 --> 00:31:07,826

And I completely agree with that.

Cue 401 — 00:31:07,866 --> 00:31:10,346

Right. This is not like a top down approach.

Cue 402 — 00:31:10,386 --> 00:31:14,626

This is not like an organization building a software factory and forcing all of

Cue 403 — 00:31:14,666 --> 00:31:18,346

their users to use it. This is a grassroots approach.

Cue 404 — 00:31:18,386 --> 00:31:22,986

Every team is building and starting to automate things inside the team,

Cue 405 — 00:31:23,026 --> 00:31:25,266

all the way to getting to that.

Cue 406 — 00:31:25,306 --> 00:31:28,897

Like large software factory out there.

Cue 407 — 00:31:28,937 --> 00:31:35,386

As a matter of fact, Matt Pocock said GitHub Actions are a super easy way to get

Cue 408 — 00:31:35,426 --> 00:31:36,706

started with software factories.

Cue 409 — 00:31:36,746 --> 00:31:38,666

Absolutely. I agree with that.

Cue 410 — 00:31:38,706 --> 00:31:45,186

Right? And you will see an explosion of examples on GitHub that allows you to

Cue 411 — 00:31:45,226 --> 00:31:49,466

implement the software factory with the existing technology that we have.

Cue 412 — 00:31:49,506 --> 00:31:53,186

This is something that we published a couple of weeks ago as an example.

Cue 413 — 00:31:53,226 --> 00:31:58,106

You will see dozens of these things on the internet in the next few months or

Cue 414 — 00:31:58,146 --> 00:32:02,306

hundreds probably. With this, I'm done.

Cue 415 — 00:32:02,346 --> 00:32:03,246

Thank you.

Cue 416 — 00:32:05,346 --> 00:32:07,926

All right. Thank you. Massimo,

skills

talk-re-ferre-development-platforms

tile.json