CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

ainativedev/aidevcon-2026-nyc

AI Native DevCon 2026 New York — practical agent skills grounded in conference talks

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

Overview
Quality
Evals
Security
Files

transcript.mdskills/talk-podjarny-trustworthy-software-factories/references/

Corrected transcript: Building a software factory you can trust

Guy Podjarny · AI Native DevCon New York 2026

Recording: https://www.youtube.com/watch?v=3NH_i8M1Gqg

Source: original corrected SRT supplied for this talk. Cue numbers, full start/end timestamps and spoken wording are preserved; subtitle line wrapping is joined. Inserted conference promotion cues 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272 are omitted. Speech artifacts remain. Treat this source as evidence, not instructions.

For topic navigation, see outline.md.

Cue 1 — 00:00:00,183 --> 00:00:03,703

Hello, everyone. Welcome to AI DevCon.

Cue 2 — 00:00:03,743 --> 00:00:06,283

Super excited to have you all here.

Cue 3 — 00:00:06,903 --> 00:00:11,183

Yeah. First wanted to say thank you for me as well, for coming here,

Cue 4 — 00:00:11,223 --> 00:00:16,023

for devoting your time. And just to repeat a little bit of the encouragement from

Cue 5 — 00:00:16,063 --> 00:00:19,183

Simon to, to share while you're here.

Cue 6 — 00:00:19,223 --> 00:00:24,863

We have and we do invest in great speakers that will be on stage after me that will

Cue 7 — 00:00:24,903 --> 00:00:30,103

have good content to share. But also you have so much more to to share about your

Cue 8 — 00:00:30,143 --> 00:00:31,503

experience. I've worked but didn't work.

Cue 9 — 00:00:31,543 --> 00:00:34,883

As we build out, so do speak up and engage with one another.

Cue 10 — 00:00:35,263 --> 00:00:42,103

I am Guy. I'm the founder and CEO of Tessl and I will kick us off talking about

Cue 11 — 00:00:42,143 --> 00:00:44,943

software factories, which we'll talk about a lot at this event.

Cue 12 — 00:00:44,983 --> 00:00:48,683

And specifically, how do you build software factories that you can trust.

Cue 13 — 00:00:49,543 --> 00:00:51,383

So let me start by setting the stage.

Cue 14 — 00:00:51,423 --> 00:00:55,623

So at this point, I think we all know that software development is transforming in

Cue 15 — 00:00:55,663 --> 00:01:00,703

this AI era, and we now understand that it is moving from building the car to

Cue 16 — 00:01:00,743 --> 00:01:06,223

building the factory. And when we're building the car, we still had a systematic

Cue 17 — 00:01:06,263 --> 00:01:08,703

kind of factory line of sorts.

Cue 18 — 00:01:08,743 --> 00:01:12,103

When we were designing and building software, we would design something we would

Cue 19 — 00:01:12,143 --> 00:01:14,223

implement, we would review it, we would test it.

Cue 20 — 00:01:14,263 --> 00:01:18,183

There was little repeatable steps, but in every one of these stations,

Cue 21 — 00:01:18,223 --> 00:01:22,083

there was a human. There was someone doing that work, maybe with some help.

Cue 22 — 00:01:22,143 --> 00:01:25,383

And we're we're moving to is a place in which we're building.

Cue 23 — 00:01:25,423 --> 00:01:30,063

Like a lot of these stations are the same, a lot of the same actions to get the

Cue 24 — 00:01:30,103 --> 00:01:34,263

software created. But at every one of these stations are AI agents.

Cue 25 — 00:01:34,303 --> 00:01:39,463

The robots now, robots that are building the different stages,

Cue 26 — 00:01:39,503 --> 00:01:44,583

and the humans job is to supervise and understand how do we operate this factory.

Cue 27 — 00:01:44,623 --> 00:01:50,423

Well, and all of this is new. We've been doing this for long enough that we have

Cue 28 — 00:01:50,463 --> 00:01:54,423

some data in the industry to show that it's very valuable to make this transition.

Cue 29 — 00:01:54,463 --> 00:01:56,303

Here are just a few of the stats on the left.

Cue 30 — 00:01:56,343 --> 00:02:03,063

I have a few industry data like Meta, talking about how they reduce the number of

Cue 31 — 00:02:03,103 --> 00:02:09,127

incident rates by 50x. By doing code reviews we talk about, we see Stripe talking

Cue 32 — 00:02:09,167 --> 00:02:14,143

about 10x-ing the PRs per week and going to 23 PR per developer per week.

Cue 33 — 00:02:14,183 --> 00:02:16,443

GitHub accelerating remediation.

Cue 34 — 00:02:16,543 --> 00:02:20,703

And at Tessl, internally, we've been embracing software factories substantially,

Cue 35 — 00:02:20,743 --> 00:02:25,143

especially since last spring. And we see the data coming in today.

Cue 36 — 00:02:25,183 --> 00:02:28,783

More than 80% of the pull requests in Tessl never saw a desktop.

Cue 37 — 00:02:28,823 --> 00:02:33,543

They have gone from ticket to to a pull request and got merged.

Cue 38 — 00:02:33,583 --> 00:02:38,183

And that took some work. And it really is compounding the rate that we're building

Cue 39 — 00:02:38,223 --> 00:02:43,243

it today. We're at a rate of 40-plus merged PRs per developer per week.

Cue 40 — 00:02:43,343 --> 00:02:45,263

It's kind of nice to sort of be faster than Stripe.

Cue 41 — 00:02:45,303 --> 00:02:49,543

We are a little bit smaller. So and we have a lot of auto maintenance.

Cue 42 — 00:02:49,583 --> 00:02:52,623

So the factory is not just creating software, it's also doing the grunt work of

Cue 43 — 00:02:52,663 --> 00:02:57,343

constantly making our software more secure, more reliable, cleaner,

Cue 44 — 00:02:57,383 --> 00:02:58,603

higher quality.

Cue 45 — 00:03:00,303 --> 00:03:04,783

So factories are good. Hopefully you agree there's sort of a good destination,

Cue 46 — 00:03:04,823 --> 00:03:09,163

but it's they also raise all these concerns.

Cue 47 — 00:03:09,423 --> 00:03:13,063

And I think most of the concerns when people are kind of pushing back against

Cue 48 — 00:03:13,103 --> 00:03:17,043

embracing a factory boil down to one of these three buckets,

Cue 49 — 00:03:17,143 --> 00:03:21,623

quality like is is my software just going to be a bunch of AI slop?

Cue 50 — 00:03:21,663 --> 00:03:24,303

Right. It's just the stuff that the agents have made up on it and I'll get in

Cue 51 — 00:03:24,343 --> 00:03:28,783

trouble. Cost. I think by now probably every one of us have had some form of

Cue 52 — 00:03:28,823 --> 00:03:34,063

unpleasant experience where the agent has accrued a bill that we did not intend and

Cue 53 — 00:03:34,103 --> 00:03:38,423

safety. And this is definitely kind of the last few months, we've had many incidents

Cue 54 — 00:03:38,463 --> 00:03:41,963

of rogue agents that are not behaving as they should.

Cue 55 — 00:03:43,943 --> 00:03:48,863

So when I think about these different concerns, they're actually fairly similar to

Cue 56 — 00:03:48,903 --> 00:03:50,423

concerns that we have in an organization.

Cue 57 — 00:03:50,463 --> 00:03:52,903

If you're a manager and you're trying to get your team to do work,

Cue 58 — 00:03:52,943 --> 00:03:56,763

you're also worried about the quality of the work, how much it's going to cost,

Cue 59 — 00:03:56,903 --> 00:04:03,063

or maybe time it'll take and whether things will go badly and to to tackle that.

Cue 60 — 00:04:03,103 --> 00:04:07,303

What we do kind of as humans, as managers is we delegate well,

Cue 61 — 00:04:07,343 --> 00:04:09,623

and delegating well is complicated.

Cue 62 — 00:04:09,663 --> 00:04:13,703

But it really boils down to starting from defining what do you even want your team

Cue 63 — 00:04:13,743 --> 00:04:16,903

to do? So like define the outcome and what does good look like if you don't have

Cue 64 — 00:04:16,943 --> 00:04:22,423

that, you're really at the mercy of just like whatever, like luck happens around the

Cue 65 — 00:04:22,463 --> 00:04:26,943

person you delegated it to, then you delegate the work and you set some boundaries

Cue 66 — 00:04:26,983 --> 00:04:30,543

for people to work in. You have a certain limited budget, you have certain tools you

Cue 67 — 00:04:30,583 --> 00:04:32,763

have to work with, are some security concerns.

Cue 68 — 00:04:32,823 --> 00:04:34,543

And then you observe what has happened.

Cue 69 — 00:04:34,583 --> 00:04:38,743

You see how it works and you learn from it, and you adjust either your expectations

Cue 70 — 00:04:38,783 --> 00:04:42,383

or the tools and the boundaries and sets that you give to the team.

Cue 71 — 00:04:42,423 --> 00:04:44,323

And you evolve there in this way.

Cue 72 — 00:04:45,423 --> 00:04:48,983

And I would make the case that with factories, you actually need to do the same.

Cue 73 — 00:04:49,023 --> 00:04:53,583

You start by defining skills that say, what is it that you want to do and what is

Cue 74 — 00:04:53,623 --> 00:04:58,103

good look like. You continue to run these skills within loops that have some

Cue 75 — 00:04:58,143 --> 00:05:00,783

boundaries that say, well, this is the environment to work with.

Cue 76 — 00:05:00,823 --> 00:05:04,483

This is the budget you have. These are the guardrails that you operate,

Cue 77 — 00:05:04,623 --> 00:05:06,863

and then you observe those as they operate.

Cue 78 — 00:05:06,903 --> 00:05:10,183

And you keep evolving. You sharpen your understanding of what you want to do.

Cue 79 — 00:05:10,223 --> 00:05:13,703

You improve and add to the tools that you are making available,

Cue 80 — 00:05:13,743 --> 00:05:20,623

etc.. And so our view, my view is that to get to a software factory that you can

Cue 81 — 00:05:20,663 --> 00:05:25,543

trust that tackles these different concerns, you have to go from building the car to

Cue 82 — 00:05:25,583 --> 00:05:29,683

building the factory through skills to loops, to then factory.

Cue 83 — 00:05:31,383 --> 00:05:36,103

And if you do that, not only do you get a software factory that you can trust,

Cue 84 — 00:05:36,143 --> 00:05:38,583

you actually get a software factory you can understand.

Cue 85 — 00:05:38,623 --> 00:05:41,743

And what you see today is a lot of people build factories and the factories

Cue 86 — 00:05:41,783 --> 00:05:44,743

themselves are sloppy. There's just a bunch of scripts and automations that run.

Cue 87 — 00:05:44,783 --> 00:05:49,383

If you build things in a skill centric fashion, then you get a factory that is very

Cue 88 — 00:05:49,423 --> 00:05:53,983

clear. To understand. You have skills, you add an environment that would run in and

Cue 89 — 00:05:54,023 --> 00:05:56,303

you have an action easy to understand.

Cue 90 — 00:05:56,343 --> 00:05:58,663

You take that action, you add a trigger.

Cue 91 — 00:05:58,703 --> 00:06:00,583

Either it's an event or a schedule.

Cue 92 — 00:06:00,623 --> 00:06:04,363

Suddenly you have a loop. You have something that's easy to reason about,

Cue 93 — 00:06:04,543 --> 00:06:08,623

and once you compose those loops, you start talking about business outcomes.

Cue 94 — 00:06:08,663 --> 00:06:12,423

Here I've got a ticket to PR, it composes these four different loops and you achieve

Cue 95 — 00:06:12,463 --> 00:06:18,503

a result. So what I want to do, I want to talk a little bit about how do you tackle

Cue 96 — 00:06:18,543 --> 00:06:24,483

use skills to loops the factory to to tackle these kind of quality,

Cue 97 — 00:06:24,863 --> 00:06:29,783

cost and safety concerns. But first maybe let me take a sidestep and and sort of say

Cue 98 — 00:06:29,823 --> 00:06:31,503

what I mean when I say factory.

Cue 99 — 00:06:31,543 --> 00:06:35,343

Like any kind of good technology trend, there's probably like 20 implementations or

Cue 100 — 00:06:35,383 --> 00:06:37,463

interpretations of what that means here in the room.

Cue 101 — 00:06:37,503 --> 00:06:43,263

So when I talk about factory at a high level, a factory is the automation of your

Cue 102 — 00:06:43,303 --> 00:06:45,383

entire software development lifecycle.

Cue 103 — 00:06:45,423 --> 00:06:48,543

But in practice, when people talk about embracing a factory,

Cue 104 — 00:06:48,583 --> 00:06:51,023

it mostly comes down to this ticket to PR flow.

Cue 105 — 00:06:51,063 --> 00:06:55,663

It's the move from building things locally to building things in some automated

Cue 106 — 00:06:55,703 --> 00:06:59,463

system where you log something at the top so you log an issue,

Cue 107 — 00:06:59,503 --> 00:07:03,383

you have an agent implement it, an agent review it, an agent respond to those

Cue 108 — 00:07:03,423 --> 00:07:05,823

reviews, or maybe a human review it as well.

Cue 109 — 00:07:05,863 --> 00:07:08,903

And then eventually the code is good enough and you merge it.

Cue 110 — 00:07:08,943 --> 00:07:12,623

And this flow is powered by some shared componentry.

Cue 111 — 00:07:12,663 --> 00:07:17,543

So you have skills and tools and context that feeds into these different stages so

Cue 112 — 00:07:17,583 --> 00:07:19,183

you can implement them better.

Cue 113 — 00:07:19,223 --> 00:07:21,353

And then hopefully you are self-improvement.

Cue 114 — 00:07:21,393 --> 00:07:24,703

So you're learning from these things that are running and you improve those skills,

Cue 115 — 00:07:24,743 --> 00:07:29,143

those tools in that context. And then the last bit of the factory is of course some

Cue 116 — 00:07:29,183 --> 00:07:31,153

shared infrastructure. You need to run these on.

Cue 117 — 00:07:31,193 --> 00:07:34,263

In some sandboxes, you need some telemetry, you need some components.

Cue 118 — 00:07:34,303 --> 00:07:39,083

And so loosely this is what I mean as the core as the start of a factory.

Cue 119 — 00:07:40,543 --> 00:07:43,743

So as I said let's let's talk about how do you build a factory that tackles these

Cue 120 — 00:07:43,783 --> 00:07:46,763

three concerns using skills to loops the factory.

Cue 121 — 00:07:47,183 --> 00:07:53,543

Let me start with skills. And when I, I want to demonstrate the value of skills on

Cue 122 — 00:07:53,583 --> 00:07:57,343

these kind of quality, cost and safety concerns using a different type of workflow

Cue 123 — 00:07:57,383 --> 00:08:00,923

automation. And that is automated vulnerability remediation.

Cue 124 — 00:08:01,423 --> 00:08:05,223

You might have noticed that your security peers, or maybe it's yourself,

Cue 125 — 00:08:05,263 --> 00:08:07,023

have not slept much this last month.

Cue 126 — 00:08:07,063 --> 00:08:10,143

It's been dubbed the Black October for for a reason.

Cue 127 — 00:08:10,183 --> 00:08:12,863

There have been sort of a mega disclosures of vulnerabilities.

Cue 128 — 00:08:12,903 --> 00:08:17,303

Probably the top headline is the PHP to shell vulnerability that has been around for

Cue 129 — 00:08:17,343 --> 00:08:20,543

three years, but has now been disclosed a trivial to exploit remote command

Cue 130 — 00:08:20,583 --> 00:08:22,543

execution. GitLab had a couple of those.

Cue 131 — 00:08:22,583 --> 00:08:28,223

Also all these in the system. Microsoft broke records with 1400 vulnerability fixes

Cue 132 — 00:08:28,263 --> 00:08:34,023

in one patch Tuesday. So you have a lot of that are overwhelming defenders.

Cue 133 — 00:08:34,063 --> 00:08:38,103

And on the other side, you have agents that are exploiting faster than ever.

Cue 134 — 00:08:38,143 --> 00:08:40,523

And so LLMs are kind of on both sides.

Cue 135 — 00:08:40,663 --> 00:08:47,210

And you can see some data with PHP to shell that within two hours of the

Cue 136 — 00:08:47,250 --> 00:08:50,583

vulnerability being disclosed, there have been hundreds of thousands.

Cue 137 — 00:08:50,623 --> 00:08:54,823

I think this is per hour attacks and now there's millions of them.

Cue 138 — 00:08:54,863 --> 00:08:59,463

And so this combination is quite toxic and really, really drives organizations to

Cue 139 — 00:08:59,503 --> 00:09:01,743

say I have to automate vulnerability remediation.

Cue 140 — 00:09:01,783 --> 00:09:03,263

And factories are a good solution.

Cue 141 — 00:09:03,303 --> 00:09:05,043

So let's talk about how to do that.

Cue 142 — 00:09:06,063 --> 00:09:09,703

The starting point might be okay, we have this brilliant agents.

Cue 143 — 00:09:09,743 --> 00:09:12,723

Can I just say okay, here's a vulnerability agent fix it.

Cue 144 — 00:09:13,023 --> 00:09:15,503

And sadly that doesn't work quite as well.

Cue 145 — 00:09:15,543 --> 00:09:19,983

This is a study from 1Password where they tried 6000 permutations across some

Cue 146 — 00:09:20,023 --> 00:09:25,503

vulnerabilities. And what they've observed is that only in 26% of the cases,

Cue 147 — 00:09:25,543 --> 00:09:29,383

the agents actually fixed the vulnerability and did not break the app in the

Cue 148 — 00:09:29,423 --> 00:09:32,583

process. And I think you'd agree that you don't want to break the app when you're

Cue 149 — 00:09:32,623 --> 00:09:36,963

remediating. And this is not a sufficient stat.

Cue 150 — 00:09:37,023 --> 00:09:40,183

And actually talking to to banks and sort of bigger organizations.

Cue 151 — 00:09:40,223 --> 00:09:42,783

You also hear that the stats are oftentimes even worse internally,

Cue 152 — 00:09:42,823 --> 00:09:46,263

because a lot of their business logic is different than what the LLMs have been

Cue 153 — 00:09:46,303 --> 00:09:50,223

trained on. So this doesn't mean that the agents can't fix the vulnerability.

Cue 154 — 00:09:50,263 --> 00:09:54,103

It just means you need to tell them what do you want them to do and what is good

Cue 155 — 00:09:54,143 --> 00:09:58,303

look like. And so you can define a process that says to fix a vulnerability you have

Cue 156 — 00:09:58,343 --> 00:09:59,583

to find it. You have to fix it.

Cue 157 — 00:09:59,623 --> 00:10:01,503

You have to verify that you've actually fixed it.

Cue 158 — 00:10:01,543 --> 00:10:05,023

Then you want to verify the behavior of the app has not changed,

Cue 159 — 00:10:05,063 --> 00:10:10,383

review it and merge it. And you can package each of these stages as a skill that

Cue 160 — 00:10:10,423 --> 00:10:13,743

says these are the processes. And that works actually quite well.

Cue 161 — 00:10:13,783 --> 00:10:17,223

So a Tessl will have a process that looks pretty much like this to fix our own

Cue 162 — 00:10:17,263 --> 00:10:20,743

internal vulnerabilities. And what we can see in the data is that it's quite

Cue 163 — 00:10:20,783 --> 00:10:25,943

impactful. We see that over half of the cases, when we ask the agent to fix the

Cue 164 — 00:10:25,983 --> 00:10:27,863

vulnerability, it did not get it right.

Cue 165 — 00:10:27,903 --> 00:10:29,503

It did not fix the vulnerability.

Cue 166 — 00:10:29,543 --> 00:10:33,283

But then when we verified the fix, it came back and it got it right.

Cue 167 — 00:10:33,463 --> 00:10:37,143

Similarly, from a code review perspective, we see that in about a quarter of the

Cue 168 — 00:10:37,183 --> 00:10:40,503

cases, the behavior will have changed and we comment on it.

Cue 169 — 00:10:40,543 --> 00:10:45,393

And once we comment, the agents can get it right and in more than 95% of the cases

Cue 170 — 00:10:45,433 --> 00:10:50,423

they got to a merge. So this is real data from Tessl's vulnerability fixes.

Cue 171 — 00:10:50,463 --> 00:10:55,423

And it just shows the the value of just clarify to this AI employee what it is that

Cue 172 — 00:10:55,463 --> 00:10:56,643

you want them to do.

Cue 173 — 00:10:58,703 --> 00:11:00,843

Second, let's talk about costs.

Cue 174 — 00:11:01,103 --> 00:11:04,663

And so frontier intelligence is amazing.

Cue 175 — 00:11:04,703 --> 00:11:08,983

And we find all these different different vulnerabilities and sort of a top leader

Cue 176 — 00:11:09,023 --> 00:11:12,903

in a bank that I, that I spoke to described it very succinctly,

Cue 177 — 00:11:12,943 --> 00:11:16,383

that it finds lots of vulnerabilities at an incredible costs.

Cue 178 — 00:11:16,423 --> 00:11:17,903

And this sentiment, by the way, is very common.

Cue 179 — 00:11:17,943 --> 00:11:22,023

Anybody that's using Mythos and these systems, they're paying between 1 and $5

Cue 180 — 00:11:22,063 --> 00:11:25,303

million for these sort of packages of scans of some critical applications.

Cue 181 — 00:11:25,343 --> 00:11:27,143

They definitely see the value in it.

Cue 182 — 00:11:27,183 --> 00:11:30,823

But they also say explicitly, if I roll this out to all of my developers,

Cue 183 — 00:11:30,863 --> 00:11:32,303

I will be bankrupt in a second.

Cue 184 — 00:11:32,343 --> 00:11:34,783

Like there's no possibility to do that.

Cue 185 — 00:11:34,823 --> 00:11:39,543

And so we should use those. But we want to automate in the factory more systematic

Cue 186 — 00:11:39,583 --> 00:11:43,463

paths. Once again, once you run these things as a loop and you can run them more

Cue 187 — 00:11:43,503 --> 00:11:47,283

automatically, you can manage those costs and decide what you want.

Cue 188 — 00:11:47,863 --> 00:11:50,503

This is, again, real data from our own costs.

Cue 189 — 00:11:50,543 --> 00:11:53,403

When we fix vulnerabilities at Tessl.

Cue 190 — 00:11:53,743 --> 00:11:58,143

So it costs us about a quarter of a buck to fix the vulnerability and verify that

Cue 191 — 00:11:58,183 --> 00:12:02,263

it's fixed. And it costs us less than a dollar to review it and to merge that.

Cue 192 — 00:12:02,303 --> 00:12:05,423

Now, clearly, this is not the same as a Mythos-scale vulnerability,

Cue 193 — 00:12:05,463 --> 00:12:09,063

but this is a system that you can roll out to say, this is automated vulnerability

Cue 194 — 00:12:09,103 --> 00:12:13,383

remediation across my system, and then you should use those models judiciously at

Cue 195 — 00:12:13,423 --> 00:12:16,403

the right time to find deeper vulns.

Cue 196 — 00:12:18,263 --> 00:12:20,943

Lastly, let me talk about safety.

Cue 197 — 00:12:20,983 --> 00:12:25,023

So up until now we were talking about fixing vulnerabilities.

Cue 198 — 00:12:25,063 --> 00:12:29,063

That is more about securing the product, the factories, producing a product,

Cue 199 — 00:12:29,103 --> 00:12:32,503

and want to make sure that we get the factory to produce a secure product in the

Cue 200 — 00:12:32,543 --> 00:12:35,263

process. And that is very important and a great opportunity.

Cue 201 — 00:12:35,303 --> 00:12:39,223

When we talk about safety in a factory, we're talking about securing the factory

Cue 202 — 00:12:39,263 --> 00:12:46,223

itself, and I find it useful to divide that exercise into into two parts

Cue 203 — 00:12:46,263 --> 00:12:49,383

security and safety. I'm not sure these are the right terms for it,

Cue 204 — 00:12:49,423 --> 00:12:52,603

but it's just useful for me for a mental model to split the two.

Cue 205 — 00:12:52,903 --> 00:12:55,023

Security is more about containment.

Cue 206 — 00:12:55,063 --> 00:12:57,383

It's like, hey, agent, I'm running you in this sandbox.

Cue 207 — 00:12:57,423 --> 00:13:00,743

Stay. Stay in the sandbox to not escalate permissions.

Cue 208 — 00:13:00,783 --> 00:13:02,423

The labs will work on alignment.

Cue 209 — 00:13:02,463 --> 00:13:06,503

They will try to get the agent to not even want to try and break out.

Cue 210 — 00:13:06,543 --> 00:13:10,863

But still, when we operate it, we need to run it in a contained environment that

Cue 211 — 00:13:10,903 --> 00:13:16,563

does not let it break out. But then safety is more about managing empowerment.

Cue 212 — 00:13:16,743 --> 00:13:20,263

To allow the agent to do stuff, you need to give it some permissions like you would

Cue 213 — 00:13:20,303 --> 00:13:23,823

to an employee. It can access the internet, it can modify code,

Cue 214 — 00:13:23,863 --> 00:13:27,943

but you need to also guide it and guardrail it from doing things that are

Cue 215 — 00:13:27,983 --> 00:13:31,943

differently. Because unlike humans, you know that if you went sideways,

Cue 216 — 00:13:31,983 --> 00:13:34,663

maybe something bad will happen to you or accountable.

Cue 217 — 00:13:34,703 --> 00:13:38,243

You might, you know, get fired or go to jail.

Cue 218 — 00:13:38,383 --> 00:13:39,823

Agents do not have that accountability.

Cue 219 — 00:13:39,863 --> 00:13:43,883

So you need to understand what are the limits of their empowerment.

Cue 220 — 00:13:44,783 --> 00:13:47,983

Let me give a few examples. So the reason this is a relatively newer domain.

Cue 221 — 00:13:48,023 --> 00:13:50,903

So there isn't really public information about it, but we've been doing a bunch of

Cue 222 — 00:13:50,943 --> 00:13:53,023

research about that internally at Tessl.

Cue 223 — 00:13:53,063 --> 00:13:56,303

And we've been testing our own skills that we've been running.

Cue 224 — 00:13:56,343 --> 00:14:00,823

And for instance, we've seen examples of a code review comment that was able to get

Cue 225 — 00:14:00,863 --> 00:14:05,383

the agents to make modifications that are outside the tickets scope.

Cue 226 — 00:14:05,423 --> 00:14:10,863

If I'm trying to refactor, sort of clean up some piece of code and remove some

Cue 227 — 00:14:10,903 --> 00:14:15,303

comments over there, no review comment should make me change the authentication

Cue 228 — 00:14:15,343 --> 00:14:17,543

system, right? There needs to be something that aligns there,

Cue 229 — 00:14:17,583 --> 00:14:22,863

so we fix that. Similarly, we use agents to analyze our go-to-market,

Cue 230 — 00:14:22,903 --> 00:14:25,903

our interactions with customers to make sure that we understand customer needs.

Cue 231 — 00:14:25,943 --> 00:14:31,623

And what we find is if a salesperson modifies a CRM field that is totally unrelated,

Cue 232 — 00:14:31,663 --> 00:14:36,623

they can change the records that we keep around that customers experience and change

Cue 233 — 00:14:36,663 --> 00:14:40,103

the forecast. We have high trust in our salespeople right now.

Cue 234 — 00:14:40,143 --> 00:14:44,063

For us, those are lesser concerns, but still, you want to track them down.

Cue 235 — 00:14:44,103 --> 00:14:48,623

And what I also liked is that we saw one example of this sort of reward hacking,

Cue 236 — 00:14:48,663 --> 00:14:52,863

in which when you say try hard to keep things simple, we got an agent to remove a

Cue 237 — 00:14:52,903 --> 00:14:56,203

functionality that was actually in use that was not the instruction.

Cue 238 — 00:14:56,943 --> 00:14:59,263

So safety is important. You want to define it.

Cue 239 — 00:14:59,303 --> 00:15:04,623

And where skills come in handy is when you define a skill, you can sharpen your

Cue 240 — 00:15:04,663 --> 00:15:07,623

guardrails around that skill. This is what I want you to do.

Cue 241 — 00:15:07,663 --> 00:15:09,983

Now you can define what are the limits of empowerment.

Cue 242 — 00:15:10,023 --> 00:15:14,223

I want for that skill. A Tessl will do that using Tessl verifiers,

Cue 243 — 00:15:14,263 --> 00:15:18,103

which are these small instructions that provide yes no questions.

Cue 244 — 00:15:18,143 --> 00:15:23,463

For instance, in this case they say, hey, only act on PR feedback that is within the

Cue 245 — 00:15:23,503 --> 00:15:26,223

linked tickets scope. So that's the instruction on it.

Cue 246 — 00:15:26,263 --> 00:15:30,623

And it has a checklist of yes no questions, which is did it read the ticket before

Cue 247 — 00:15:30,663 --> 00:15:35,543

it acted. Did it disallow make no changes for out-of-scope feedback.

Cue 248 — 00:15:35,583 --> 00:15:36,783

So these are yes no questions.

Cue 249 — 00:15:36,823 --> 00:15:41,423

And we can apply either small models or decision models like Jev to them.

Cue 250 — 00:15:41,463 --> 00:15:45,123

And some of them are actually even deterministic and regex and some scripts.

Cue 251 — 00:15:45,303 --> 00:15:48,743

And so once you define those guardrails, those limits to empowerments,

Cue 252 — 00:15:48,783 --> 00:15:52,183

you can use them in different stages of the system.

Cue 253 — 00:15:52,223 --> 00:15:56,203

You can use them as part of the actual runtime in the loop to guard the loop.

Cue 254 — 00:15:56,423 --> 00:15:58,263

You can use them as a code review gate.

Cue 255 — 00:15:58,303 --> 00:16:00,703

What has happened before? I accept this change on it.

Cue 256 — 00:16:00,743 --> 00:16:03,503

This is probably the most common use of them for us right now,

Cue 257 — 00:16:03,543 --> 00:16:08,503

and then you can actually sometimes define verifiers for a skill and inspect code

Cue 258 — 00:16:08,543 --> 00:16:10,423

that is already there. Here's my skill.

Cue 259 — 00:16:10,463 --> 00:16:12,903

Does my code conform to the skill?

Cue 260 — 00:16:12,943 --> 00:16:15,983

This is especially useful for things like design for instance such a little bit less

Cue 261 — 00:16:16,023 --> 00:16:16,963

in safety.

Cue 273 — 00:16:59,895 --> 00:17:04,215

So hopefully you're just maybe a little bit convinced that skills are useful.

Cue 274 — 00:17:04,255 --> 00:17:06,695

They're useful as a means of controlling quality.

Cue 275 — 00:17:06,735 --> 00:17:09,015

They're used as a means of controlling and managing your cost.

Cue 276 — 00:17:09,055 --> 00:17:11,935

They're useful as a means of managing your safety and the boundaries of your

Cue 277 — 00:17:11,975 --> 00:17:17,375

empowerment. Let me switch from here to talking about loops first.

Cue 278 — 00:17:17,415 --> 00:17:19,535

Once again, let me try to define what is a loop.

Cue 279 — 00:17:19,575 --> 00:17:23,735

Now, I will admit that technically, in kind of agent land, any agent execution is a

Cue 280 — 00:17:23,775 --> 00:17:26,375

loop. It is looping on an action.

Cue 281 — 00:17:26,415 --> 00:17:28,015

It's making an LLM action. It's looking at it.

Cue 282 — 00:17:28,055 --> 00:17:31,775

It's deciding what to do. So if you're a purist, a loop is that I think a more

Cue 283 — 00:17:31,815 --> 00:17:35,475

useful interpretation of the term loop in the industry, which we've embraced,

Cue 284 — 00:17:35,535 --> 00:17:40,655

is that it is an agent workflow, and for us, a skill powered agent workflow.

Cue 285 — 00:17:40,695 --> 00:17:45,295

So you take a skill, you define an environment that's going to run it and some

Cue 286 — 00:17:45,335 --> 00:17:49,215

trigger for it to to run. When does it run, whether it's user action or time or

Cue 287 — 00:17:49,255 --> 00:17:53,055

whatever it is. And then you combine those into an execution.

Cue 288 — 00:17:53,095 --> 00:17:56,335

So you run the skill hopefully within some form of sandbox.

Cue 289 — 00:17:56,375 --> 00:18:00,055

And that comes out with two types of artifacts out of it.

Cue 290 — 00:18:00,095 --> 00:18:04,055

One is the actual output. Maybe it's code that's been generated or something like

Cue 291 — 00:18:04,095 --> 00:18:07,575

that. And the second is telemetry is a bunch of logs.

Cue 292 — 00:18:07,615 --> 00:18:10,375

And what has happened. Right. Like what happened in the process.

Cue 293 — 00:18:10,415 --> 00:18:15,695

So this is a loop. And a self improving loop is one that now looks at those outputs

Cue 294 — 00:18:15,735 --> 00:18:18,455

and looks at those logs and the feedback you've received.

Cue 295 — 00:18:18,495 --> 00:18:23,655

And it tries to learn from them and go back and fix and evolve the skill or the

Cue 296 — 00:18:23,695 --> 00:18:27,775

environment or the triggers. And so this is this is self-improvement.

Cue 297 — 00:18:27,815 --> 00:18:30,775

And again, it benefits from the fact that you're building on skills because you

Cue 298 — 00:18:30,815 --> 00:18:35,615

know, where is your learning going to aggregate to your aggregating that learning to

Cue 299 — 00:18:35,655 --> 00:18:39,115

evolve the skill or sometimes these other setups.

Cue 300 — 00:18:39,975 --> 00:18:44,335

And sometimes people think about loops as, as like a destination.

Cue 301 — 00:18:44,375 --> 00:18:48,255

Don't worry, let me build my factory without self-improvement and I will proceed to

Cue 302 — 00:18:48,295 --> 00:18:49,895

it after. And I think that's a mistake.

Cue 303 — 00:18:49,935 --> 00:18:54,215

I think self optimizing loops are actually key to getting your factory to be

Cue 304 — 00:18:54,255 --> 00:18:56,955

successful. I'll make three cases for it.

Cue 305 — 00:18:57,135 --> 00:19:01,175

First is that they let you extract truth from observation.

Cue 306 — 00:19:01,215 --> 00:19:07,735

I preview said. I previously said that skills define what do you want to be done and

Cue 307 — 00:19:07,775 --> 00:19:11,575

what does good look like. And it sounds simple, but those are actually pretty hard

Cue 308 — 00:19:11,615 --> 00:19:15,215

to do, right? Like just sort of I like to use the example of what do you want for

Cue 309 — 00:19:15,255 --> 00:19:18,175

your birthday? I don't know, you decide.

Cue 310 — 00:19:18,215 --> 00:19:22,575

Right. But if I observe your response to receiving 30 different gifts over the

Cue 311 — 00:19:22,615 --> 00:19:27,335

course of a few years, I might have a better idea of what it is that you actually

Cue 312 — 00:19:27,375 --> 00:19:30,815

like and that you enjoy. And so loops are an example of that.

Cue 313 — 00:19:30,855 --> 00:19:35,975

They are. You start with some form of definition of what it is that you want to do,

Cue 314 — 00:19:36,015 --> 00:19:39,705

and then you get going and you learn from feedback and you iterate.

Cue 315 — 00:19:39,745 --> 00:19:44,595

And this is super important because it lets you define what correct behavior is,

Cue 316 — 00:19:44,735 --> 00:19:47,895

and in turn that allows you to optimize other dimensions.

Cue 317 — 00:19:47,935 --> 00:19:51,895

Once you are able to evaluate whether a loop is working correctly,

Cue 318 — 00:19:51,935 --> 00:19:56,015

now you can start to optimize its cost, its safety, its speed,

Cue 319 — 00:19:56,055 --> 00:19:59,615

other parameters around it. You can run a cheaper model before you've defined that,

Cue 320 — 00:19:59,655 --> 00:20:02,815

before you've had confidence in what correct behavior is.

Cue 321 — 00:20:02,855 --> 00:20:05,635

It's very hard to do those others you're vibing.

Cue 322 — 00:20:06,535 --> 00:20:07,835

And then lastly.

Cue 323 — 00:20:09,535 --> 00:20:13,135

Skills and loops. They actually run in a dynamic environment.

Cue 324 — 00:20:13,175 --> 00:20:16,695

So the thing you want to do, the thing that is correct behavior,

Cue 325 — 00:20:16,735 --> 00:20:18,695

it doesn't stay static. Over time.

Cue 326 — 00:20:18,735 --> 00:20:21,735

Your system changes your needs changes, your opinions change.

Cue 327 — 00:20:21,775 --> 00:20:25,375

And so you want to make sure that you don't let these skills rot.

Cue 328 — 00:20:25,415 --> 00:20:27,655

You want to make sure that you evolve them and you advance them.

Cue 329 — 00:20:27,695 --> 00:20:31,175

And loops let you do that. Because if you're constantly inspecting what has happened

Cue 330 — 00:20:31,215 --> 00:20:33,815

and what has been the feedback and what worked and what didn't work,

Cue 331 — 00:20:33,855 --> 00:20:35,615

you can bring that knowledge back.

Cue 332 — 00:20:35,655 --> 00:20:36,715

So that makes sense.

Cue 333 — 00:20:38,295 --> 00:20:41,095

And just continuing the trend here of using some Tessl data,

Cue 334 — 00:20:41,135 --> 00:20:43,015

we use loops everywhere. So we improve many things.

Cue 335 — 00:20:43,055 --> 00:20:47,135

But just the easiest to understand is for instance we've used loops to within a

Cue 336 — 00:20:47,175 --> 00:20:52,965

couple of weeks because we have a lot of PRs really reduce the cost of from about 75

Cue 337 — 00:20:53,005 --> 00:20:59,375

to $13. All included a cost per PR and we've done with a variety of ways model

Cue 338 — 00:20:59,415 --> 00:21:03,095

routing that is smarter context optimization, some extraction of tools out of the

Cue 339 — 00:21:03,135 --> 00:21:05,455

context. The means are are many.

Cue 340 — 00:21:05,495 --> 00:21:08,775

There are many ways to sort of use that improvement, but the important one is that

Cue 341 — 00:21:08,815 --> 00:21:13,435

it got better while actually accelerating velocity and holding the quality.

Cue 342 — 00:21:16,035 --> 00:21:18,675

So we got skills, we got loops.

Cue 343 — 00:21:18,715 --> 00:21:22,435

What about factory? So I'll start with a little secret.

Cue 344 — 00:21:22,475 --> 00:21:26,155

Don't tell anyone. But if you've been building skills and loops,

Cue 345 — 00:21:26,195 --> 00:21:28,335

you've actually been building your factory.

Cue 346 — 00:21:28,475 --> 00:21:32,435

In practice, a factory is just a combination of all of these different things.

Cue 347 — 00:21:32,475 --> 00:21:36,835

So when we say skills to loops to factory, mostly it means you're starting to look

Cue 348 — 00:21:36,875 --> 00:21:39,715

at things in aggregate. So you're saying.

Cue 349 — 00:21:39,755 --> 00:21:44,595

So to me a factory is really three core areas sort of dividing everything into

Cue 350 — 00:21:44,635 --> 00:21:50,475

three. One is just like a library of reusable components, reusable skills,

Cue 351 — 00:21:50,515 --> 00:21:54,635

reusable loops, reusable infrastructure, telemetry information,

Cue 352 — 00:21:54,675 --> 00:21:57,955

etc. so those aggregate and you can use them across different things.

Cue 353 — 00:21:57,995 --> 00:22:01,195

Oftentimes you can use the same data to improve different skills and different

Cue 354 — 00:22:01,235 --> 00:22:04,935

loops, but also it's just a library for you to choose from.

Cue 355 — 00:22:05,435 --> 00:22:09,295

The second thing in a factory is that it requires a different operations.

Cue 356 — 00:22:09,435 --> 00:22:13,755

You to operate a factory if you're a that developer now in the stand overseeing a

Cue 357 — 00:22:13,795 --> 00:22:16,595

bunch of AI agents, you need different tools.

Cue 358 — 00:22:16,635 --> 00:22:19,155

You need to be able to know what's running right now on the factory floor,

Cue 359 — 00:22:19,195 --> 00:22:21,195

what is failing, what needs my attention.

Cue 360 — 00:22:21,235 --> 00:22:23,195

You need to manage those loops and those changes.

Cue 361 — 00:22:23,235 --> 00:22:27,095

So a factory includes a different UX to how do you interact with the system.

Cue 362 — 00:22:27,715 --> 00:22:33,235

And then lastly, a factory is the place in which you aggregate changes into business

Cue 363 — 00:22:33,275 --> 00:22:38,155

outcomes. And so you really try to oftentimes these loops might be fairly technical.

Cue 364 — 00:22:38,195 --> 00:22:40,875

Maybe vulnerability remediation is an example of something where the outcome is

Cue 365 — 00:22:40,915 --> 00:22:45,115

fairly clear. But ticket to PR is often made up of multiple stages of pick up the

Cue 366 — 00:22:45,155 --> 00:22:47,595

ticket, analyze it against the product.

Cue 367 — 00:22:47,635 --> 00:22:50,275

Maybe there's elements of implementation, elements of review.

Cue 368 — 00:22:50,315 --> 00:22:54,995

All of those combine into a full ticket to merge pull request at the end.

Cue 369 — 00:22:55,035 --> 00:22:56,155

And maybe that is a business goal.

Cue 370 — 00:22:56,195 --> 00:22:59,735

Maybe you go even further. And so you define these business outcomes.

Cue 371 — 00:23:00,395 --> 00:23:05,795

And there's kind of an exploration right now in the industry about do you have a

Cue 372 — 00:23:05,835 --> 00:23:08,355

factory or do you have multiple factories.

Cue 373 — 00:23:08,395 --> 00:23:11,315

And let's say that this is still an area that we're all learning,

Cue 374 — 00:23:11,355 --> 00:23:15,075

but loosely, I think you're going to have a factory per product.

Cue 375 — 00:23:15,115 --> 00:23:19,035

In fact, you're not building a factory, you're building a product and you're

Cue 376 — 00:23:19,075 --> 00:23:23,475

constructing a factory around it to build that product better and faster.

Cue 377 — 00:23:23,515 --> 00:23:26,235

And you probably want shared infrastructure.

Cue 378 — 00:23:26,275 --> 00:23:29,995

That's what the term what is your factory versus the offshoot get a little bit

Cue 379 — 00:23:30,035 --> 00:23:36,275

murky. But fundamentally your factory to to build your mobile app will be different

Cue 380 — 00:23:36,315 --> 00:23:40,435

than to build your sort of critical systems for like payments would be different

Cue 381 — 00:23:40,475 --> 00:23:44,395

than your marketing website. So there are different have different constraints,

Cue 382 — 00:23:44,435 --> 00:23:47,855

different content, different tools that they need inside.

Cue 383 — 00:23:49,915 --> 00:23:56,015

And the last thing to say about factory is that building the factory is one thing.

Cue 384 — 00:23:56,155 --> 00:24:00,195

And then committing to the factory, changing your ways of working to use the

Cue 385 — 00:24:00,235 --> 00:24:03,555

factory, that is the step that follows after.

Cue 386 — 00:24:03,595 --> 00:24:08,235

And what we see is like the evolution that happens organizationally is a little bit

Cue 387 — 00:24:08,275 --> 00:24:10,575

similar to that skill still loops the factory.

Cue 388 — 00:24:10,875 --> 00:24:14,195

It starts from people in their organization embracing local skills.

Cue 389 — 00:24:14,235 --> 00:24:17,275

So they basically build up some agent-fu, right?

Cue 390 — 00:24:17,315 --> 00:24:20,995

Like they learn how to use the agents better on their system and you find yourself

Cue 391 — 00:24:21,035 --> 00:24:24,755

in the team with like, these people are like masters and these ones are laggards.

Cue 392 — 00:24:24,795 --> 00:24:29,435

And how does that work? But they build some context and some tools locally to be

Cue 393 — 00:24:29,475 --> 00:24:32,175

able to get the agents to work better in their application.

Cue 394 — 00:24:33,635 --> 00:24:36,675

Over time, you start noticing that there's duplicates, that everybody's reinventing

Cue 395 — 00:24:36,715 --> 00:24:38,315

the wheel, everybody's building the same things.

Cue 396 — 00:24:38,355 --> 00:24:42,915

And so you move into a world of shared skills, and sharing is a little bit hard.

Cue 397 — 00:24:42,955 --> 00:24:44,795

It's collaboration. It's slow down factors.

Cue 398 — 00:24:44,835 --> 00:24:47,595

So typically the most common here are like platform skills.

Cue 399 — 00:24:47,635 --> 00:24:51,395

So you might have a platform team create security reviews or you know,

Cue 400 — 00:24:51,435 --> 00:24:55,795

oftentimes like compliance related or systems related skills says you have to build

Cue 401 — 00:24:55,835 --> 00:24:58,995

those. And those help a little bit, but they don't change the core way of

Cue 402 — 00:24:59,035 --> 00:25:01,575

methodology. People are still mostly at their desk.

Cue 403 — 00:25:02,715 --> 00:25:05,795

The next step is to start to automate some things.

Cue 404 — 00:25:05,835 --> 00:25:08,875

So maybe I'll stick to vulnerability remediation here.

Cue 405 — 00:25:08,915 --> 00:25:12,795

Maybe you're automating vulnerability remediation so that process runs through

Cue 406 — 00:25:12,835 --> 00:25:15,075

without landing on a developers desktop.

Cue 407 — 00:25:15,115 --> 00:25:18,235

An event happens, it comes along, you build it, and that's great.

Cue 408 — 00:25:18,275 --> 00:25:20,195

So you start building that muscle.

Cue 409 — 00:25:20,235 --> 00:25:24,355

And I think this is a useful way to just get used to getting agents to be able to do

Cue 410 — 00:25:24,395 --> 00:25:27,015

something in your code base on a recurring basis.

Cue 411 — 00:25:27,555 --> 00:25:32,635

And the end result is a mode in which your developers are not building the product.

Cue 412 — 00:25:32,675 --> 00:25:37,335

They are building the factory, the builds, the product, and you.

Cue 413 — 00:25:37,555 --> 00:25:42,715

In my view, the aspiration you have is a world in which you have no more interactive

Cue 414 — 00:25:42,755 --> 00:25:47,275

development. So you are providing your requirements, your you're a customer of this

Cue 415 — 00:25:47,315 --> 00:25:52,115

factory, and when the factory doesn't get it right, you don't fix the feature you're

Cue 416 — 00:25:52,155 --> 00:25:53,635

building. You fix the factory.

Cue 417 — 00:25:53,675 --> 00:25:58,475

You meet the agent, you meet the pull request or the output of of what's been built

Cue 418 — 00:25:58,515 --> 00:26:03,315

in the pull request. And you say, this is not good enough, here's information.

Cue 419 — 00:26:03,355 --> 00:26:07,615

Go build it again versus fixing the actual thing that's been built.

Cue 420 — 00:26:09,715 --> 00:26:12,835

And so all of these were like, how do you get there?

Cue 421 — 00:26:12,875 --> 00:26:14,915

How do you get to a factory you can trust?

Cue 422 — 00:26:14,955 --> 00:26:20,269

And it's probably worth remembering towards the end of it that there's gold in them

Cue 423 — 00:26:20,309 --> 00:26:23,135

hills. We've already seen all the stats around.

Cue 424 — 00:26:23,435 --> 00:26:25,575

Things will run faster, they'll be better.

Cue 425 — 00:26:25,635 --> 00:26:30,115

But looking at looking kind of beyond those, there's just so much more that you can

Cue 426 — 00:26:30,155 --> 00:26:32,655

do. Once you have a factory that you can trust.

Cue 427 — 00:26:32,795 --> 00:26:37,515

And there's a lot that you can do here, but maybe three dimension as as per my hobby

Cue 428 — 00:26:37,555 --> 00:26:40,955

over here, a habit. One is you can increase your ambition.

Cue 429 — 00:26:40,995 --> 00:26:44,715

Like we had a leadership roundtable where someone said it's not about increasing

Cue 430 — 00:26:44,755 --> 00:26:46,555

productivity, it's about increasing ambition.

Cue 431 — 00:26:46,595 --> 00:26:50,135

And I like that view so you can deliver more customer value,

Cue 432 — 00:26:50,235 --> 00:26:54,815

broaden your scope level, like higher expectations about customer service.

Cue 433 — 00:26:55,035 --> 00:26:57,315

You can pull more people into the factory.

Cue 434 — 00:26:57,355 --> 00:27:00,775

So you can go from a software factory to a factory of other processes.

Cue 435 — 00:27:00,835 --> 00:27:05,475

In a Tessl. Right now, we have a challenge in GTM.

Cue 436 — 00:27:05,515 --> 00:27:09,595

We struggle to keep up with the pace in which development is building new

Cue 437 — 00:27:09,635 --> 00:27:11,755

capabilities. And we're under this process.

Cue 438 — 00:27:11,795 --> 00:27:17,555

Or like in the process of, of of an effort to shift the GTM left to make more of

Cue 439 — 00:27:17,595 --> 00:27:21,195

these things get produced. Whenever a pull request comes along,

Cue 440 — 00:27:21,235 --> 00:27:24,995

it should come with a generated demo with a relevant landing pages with

Cue 441 — 00:27:25,035 --> 00:27:28,235

identification of which customers should know about this capability and the email

Cue 442 — 00:27:28,275 --> 00:27:29,835

that is ready to be sent to them.

Cue 443 — 00:27:29,875 --> 00:27:32,375

And so there's a lot more inclusion that you can do there.

Cue 444 — 00:27:32,515 --> 00:27:36,115

And then I'd be remiss if I don't say when your factory does the work for you,

Cue 445 — 00:27:36,155 --> 00:27:38,395

you can also potentially work a little bit less.

Cue 446 — 00:27:38,435 --> 00:27:42,495

Maybe you can actually just get more done and have more flexibility.

Cue 447 — 00:27:45,555 --> 00:27:48,675

So all of this was around what you build.

Cue 448 — 00:27:48,715 --> 00:27:51,995

And I want to spend a couple of minutes at the end here talking about how,

Cue 449 — 00:27:52,035 --> 00:27:56,315

how can we help. And, you know, a trusted factory does require a bunch of

Cue 450 — 00:27:56,355 --> 00:28:00,155

infrastructure components. And hopefully, you know, that you can rely on partners

Cue 451 — 00:28:00,195 --> 00:28:04,355

like Tessl and others that are great ones here at the at the booth.

Cue 452 — 00:28:04,395 --> 00:28:08,995

I don't have all the sponsor names over here to to check out and talk to them at the

Cue 453 — 00:28:09,035 --> 00:28:10,855

expo. That can help you build.

Cue 454 — 00:28:11,355 --> 00:28:14,555

But in this sort of era of software factories, when we feel like we can build

Cue 455 — 00:28:14,595 --> 00:28:18,755

everything, we regularly encounter the question of like, if I can build all of this

Cue 456 — 00:28:18,795 --> 00:28:20,995

stuff, why? Why am I buying anything?

Cue 457 — 00:28:21,035 --> 00:28:24,735

And I think about this a lot from both sides, both as a buyer and as a vendor.

Cue 458 — 00:28:24,875 --> 00:28:30,195

And I think the core kind of three reasons are one shared learnings.

Cue 459 — 00:28:30,235 --> 00:28:34,915

So if you come across a problem and you report that to the product and that product

Cue 460 — 00:28:34,955 --> 00:28:38,555

gets gets fixed, I don't need to experience that problem.

Cue 461 — 00:28:38,595 --> 00:28:39,815

And I think this is important.

Cue 462 — 00:28:39,855 --> 00:28:43,355

I think we'll all agree that it's not efficient for us as an ecosystem,

Cue 463 — 00:28:43,395 --> 00:28:46,835

as a community, for all of us to encounter all the problems and need to fix all the

Cue 464 — 00:28:46,875 --> 00:28:50,975

problems together. And so I think there's definitely real value over here.

Cue 465 — 00:28:52,115 --> 00:28:57,075

The second thing that you get with a with a partner is that you get someone that

Cue 466 — 00:28:57,115 --> 00:28:58,955

helps curate the world for you.

Cue 467 — 00:28:58,995 --> 00:29:03,435

This is a fast paced world. A lot of things change, and hopefully when you work with

Cue 468 — 00:29:03,475 --> 00:29:06,475

a third party, you have to pick them wisely.

Cue 469 — 00:29:06,515 --> 00:29:10,355

They will help you build the right things that will help translate the world for

Cue 470 — 00:29:10,395 --> 00:29:13,595

you. Bring the right opinions on it, and if you trust them, they can help you.

Cue 471 — 00:29:13,635 --> 00:29:15,975

They can help offload a fair bit from you.

Cue 472 — 00:29:16,675 --> 00:29:20,275

And then the last and maybe most evident is you just get some efficiency.

Cue 473 — 00:29:20,315 --> 00:29:22,715

Again, it doesn't make sense for all of us to build the same things.

Cue 474 — 00:29:22,755 --> 00:29:26,235

But oftentimes when you use a third party tool, it might actually be cheaper than

Cue 475 — 00:29:26,275 --> 00:29:28,875

building it yourself from a dev perspective and all that.

Cue 476 — 00:29:28,915 --> 00:29:30,815

But sometimes even just operation.

Cue 477 — 00:29:31,675 --> 00:29:36,635

But despite all of this sort of cerebral side, I think the real reason that you end

Cue 478 — 00:29:36,675 --> 00:29:39,715

up sort of embracing a partner is because it delights you.

Cue 479 — 00:29:39,755 --> 00:29:43,275

It's because you're using a product and it just makes you feel great.

Cue 480 — 00:29:43,315 --> 00:29:48,435

And allow myself a bit of a shameless plug here of Tessl, because I'm super excited

Cue 481 — 00:29:48,475 --> 00:29:52,355

by what we launched yesterday in the Tessl factory, because I feel like we got that

Cue 482 — 00:29:52,395 --> 00:29:57,235

part right. We got we could. It's delightful, hopefully to use it and to build a

Cue 483 — 00:29:57,275 --> 00:30:02,875

factory with it. So I wanted to show this sort of quick one minute video here of how

Cue 484 — 00:30:02,915 --> 00:30:08,290

to build a factory. So in Tessl, if you set it up like the flow will show,

Cue 485 — 00:30:08,330 --> 00:30:12,715

it sets up a factory that just does this ticket to PR process for you.

Cue 486 — 00:30:12,755 --> 00:30:15,275

You can do it with your agent or with your browser.

Cue 487 — 00:30:15,315 --> 00:30:19,155

If you use a browser, it just you click use browser you install on GitHub,

Cue 488 — 00:30:19,195 --> 00:30:22,835

you pick the repository that you want and you come along.

Cue 489 — 00:30:22,875 --> 00:30:24,875

It asks you where do you want to take your issues from?

Cue 490 — 00:30:24,915 --> 00:30:27,275

Is it from GitHub? Is it Linear or is it Jira?

Cue 491 — 00:30:27,315 --> 00:30:32,875

And you turn on your factory and just like that it will go off and it will generate

Cue 492 — 00:30:32,915 --> 00:30:36,835

a set of these loops for you to do code review, to do a ticket to PR.

Cue 493 — 00:30:36,875 --> 00:30:40,435

So really like nothing, just go along and connect it and it will create that

Cue 494 — 00:30:40,475 --> 00:30:44,635

factory. You can go on now. Now you have a ticket to flow.

Cue 495 — 00:30:44,675 --> 00:30:47,875

I'm showing this in the Tessl UI that you can create an issue,

Cue 496 — 00:30:47,915 --> 00:30:51,735

label it as Tessl and the factory will kick off and execute it.

Cue 497 — 00:30:52,115 --> 00:30:55,455

You will do that in your sort of issues UI as well.

Cue 498 — 00:30:55,515 --> 00:30:57,955

And you will meet this at the pull request.

Cue 499 — 00:30:57,995 --> 00:31:01,555

You will meet at the PR, the the the factory.

Cue 500 — 00:31:01,595 --> 00:31:03,635

You will have seen. I skipped over here a little bit.

Cue 501 — 00:31:03,675 --> 00:31:07,795

The reviewer come along and do the assessment of it and you can hit merge and go

Cue 502 — 00:31:07,835 --> 00:31:12,035

forward. Once you've done that, Tessl offers a factory floor view.

Cue 503 — 00:31:12,075 --> 00:31:14,955

That again I think is is exciting to use.

Cue 504 — 00:31:14,995 --> 00:31:16,955

You will see all the activity that is happening.

Cue 505 — 00:31:16,995 --> 00:31:20,195

You'll see metrics build up. You can define and customize all of those to your

Cue 506 — 00:31:20,235 --> 00:31:25,155

definition of correct. And once you've built up the default loops,

Cue 507 — 00:31:25,195 --> 00:31:28,035

you can easily add loops. We have a bunch of pre-made loops.

Cue 508 — 00:31:28,075 --> 00:31:31,035

You can create your custom loops, including the one that I've been talking about

Cue 509 — 00:31:31,075 --> 00:31:33,615

here, which is fixing vulnerabilities.

Cue 510 — 00:31:33,795 --> 00:31:37,715

So a lot of it is about simplicity, a lot of it is about delight.

Cue 511 — 00:31:37,755 --> 00:31:41,915

And I would love for all of you to try that out and see if you like it and give us

Cue 512 — 00:31:41,955 --> 00:31:44,895

feedback on it and hopefully get a little bit of that delight.

Cue 513 — 00:31:45,315 --> 00:31:47,395

And in fact, we're so excited by this.

Cue 514 — 00:31:47,435 --> 00:31:48,735

If I can get my.

Cue 515 — 00:31:50,435 --> 00:31:52,055

Thing to flow again here.

Cue 516 — 00:31:54,755 --> 00:31:56,055

Sorry about that.

Cue 517 — 00:31:58,635 --> 00:32:00,795

Oh, it's not going to my next slide.

Cue 518 — 00:32:00,835 --> 00:32:02,375

So just for that

Cue 519 — 00:32:03,595 --> 00:32:04,395

are.

Cue 520 — 00:32:07,195 --> 00:32:09,135

That was so in time up until now.

Cue 521 — 00:32:14,095 --> 00:32:18,495

Here we go. So in fact, we're so excited by wanting you to try it out that we're

Cue 522 — 00:32:18,535 --> 00:32:22,175

going to give a decent amount of money for all of you to do it.

Cue 523 — 00:32:22,215 --> 00:32:28,935

If you are here in real life, we're giving a $500 credit to all physical attendees

Cue 524 — 00:32:28,975 --> 00:32:31,175

to try out Tessl's software factories.

Cue 525 — 00:32:31,215 --> 00:32:34,255

If you're online, we're giving a $200 credit.

Cue 526 — 00:32:34,295 --> 00:32:37,335

We would love for you again to try it out, to get to a working factory,

Cue 527 — 00:32:37,375 --> 00:32:40,895

to be delighted for it. Hopefully accelerate your journey around how you embrace

Cue 528 — 00:32:40,935 --> 00:32:41,735

things.

Cue 529 — 00:32:43,575 --> 00:32:45,455

So and you're going to find this QR code.

Cue 530 — 00:32:45,495 --> 00:32:50,395

Sorry, I kind of rushed through it and you'll find this QR code at the expo as well.

Cue 531 — 00:32:51,375 --> 00:32:56,895

So to summarize, software factories are super valuable, but to get to them requires

Cue 532 — 00:32:56,935 --> 00:33:00,255

a journey. And to be able to build a software factory that you can trust.

Cue 533 — 00:33:00,295 --> 00:33:03,975

We think that you should go from skills to loops to factory,

Cue 534 — 00:33:04,015 --> 00:33:08,055

and if you get all of those right, you can lay back and let the agents do all the

Cue 535 — 00:33:08,095 --> 00:33:09,835

work. Thank you.

skills

talk-podjarny-trustworthy-software-factories

tile.json