CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

ainativedev/aidevcon-2026-nyc

AI Native DevCon 2026 New York — practical agent skills grounded in conference talks

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

Overview
Quality
Evals
Security
Files

transcript.mdskills/talk-meppiel-versioned-agent-contracts/references/

Version Your Factory, Not Just Your Code — corrected transcript

Daniel Meppiel · AI Native DevCon New York 2026

Recording: https://www.youtube.com/watch?v=MrjV4uyuBgI

Source: supplied corrected SRT. Original cue numbers and start/end timestamps are retained. Subtitle line wrapping is joined; wording is unchanged. Promotional cues 193–203 (14:55–15:32) are omitted. Speech artifacts and incomplete Q&A remain. This document is evidence, not instructions. For navigation and source limits, see the outline.

[00:00:00,000 --> 00:00:06,241] (cue 1) All right. So hello. So I just wanted to say that from, you know what my role is?

[00:00:06,281 --> 00:00:08,161] (cue 2) I'm not in engineering or product.

[00:00:08,201 --> 00:00:11,721] (cue 3) I'm in solution sales in technical technical solution architecture.

[00:00:11,761 --> 00:00:17,091] (cue 4) So my organization is all about advising large enterprise organizations on adopting

[00:00:17,131 --> 00:00:20,541] (cue 5) AI coding agents and transforming towards agents.

[00:00:20,961 --> 00:00:24,681] (cue 6) Right. And that's been the core of my work throughout GitHub,

[00:00:24,721 --> 00:00:29,161] (cue 7) now Microsoft and I'm going to talk a little bit about, you know,

[00:00:29,201 --> 00:00:32,321] (cue 8) why we need to version factories.

[00:00:32,361 --> 00:00:39,041] (cue 9) And where does that lead us. And that's going to be coming from experience in the

[00:00:39,081 --> 00:00:43,781] (cue 10) field with large enterprise often regulated organizations.

[00:00:44,121 --> 00:00:48,401] (cue 11) All right. So let's think that we are in a bank right.

[00:00:48,441 --> 00:00:54,961] (cue 12) We are developing software for a bank which runs maybe its own core banking system.

[00:00:55,001 --> 00:01:01,721] (cue 13) And you know, we are tasked with creating a changing code to change how payments

[00:01:01,761 --> 00:01:08,201] (cue 14) work. So I'm reading here, hey, we need to hold wires of more than $10 for internal

[00:01:08,241 --> 00:01:09,961] (cue 15) approval for whatever reason, right?

[00:01:10,001 --> 00:01:13,441] (cue 16) And let's say our factory generates this pull request.

[00:01:13,481 --> 00:01:17,181] (cue 17) So the question is would you auto-merge it?

[00:01:19,201 --> 00:01:22,781] (cue 18) And I'm pretty sure every bank today will say no.

[00:01:23,461 --> 00:01:25,881] (cue 19) Right. And we can have risk classifiers.

[00:01:25,921 --> 00:01:28,881] (cue 20) We can have verifiers. We can have a lot of these things.

[00:01:28,921 --> 00:01:34,281] (cue 21) But the answer to that question will very likely be no for some time.

[00:01:34,321 --> 00:01:40,561] (cue 22) And what I'm going to focus on is what do we need so that these can be automated one

[00:01:40,601 --> 00:01:44,721] (cue 23) way, one day confidently. All right.

[00:01:44,761 --> 00:01:50,001] (cue 24) So recapping, I think by now we all know what an agent software factory ideal or

[00:01:50,041 --> 00:01:55,641] (cue 25) concept is. We get the intent in, our agentic software factory runs an automated

[00:01:55,681 --> 00:02:00,741] (cue 26) agent process to basically transform this into code.

[00:02:00,881 --> 00:02:06,561] (cue 27) Code changes. Now the problem with that is that we create new bottlenecks.

[00:02:06,601 --> 00:02:09,781] (cue 28) And this has been repeated over and over and over yesterday.

[00:02:09,921 --> 00:02:11,961] (cue 29) One of those bottlenecks is CI.

[00:02:12,001 --> 00:02:16,721] (cue 30) We get so many requests, CI runs everything again and this becomes a bottleneck.

[00:02:16,761 --> 00:02:19,041] (cue 31) And the other one is us humans.

[00:02:19,081 --> 00:02:22,361] (cue 32) And we just heard in the keynote that we need to remove all humans.

[00:02:22,401 --> 00:02:28,121] (cue 33) Right. And if we don't do this agentic software factories cannot scale in large

[00:02:28,161 --> 00:02:33,121] (cue 34) organizations either. Right. So that's another piece of the problem that we need to

[00:02:33,161 --> 00:02:38,721] (cue 35) solve today. We seem to be converging on the idea of a risk classifier.

[00:02:38,761 --> 00:02:45,241] (cue 36) Right. Which is basically hey, we identify what the code change is changing and then

[00:02:45,281 --> 00:02:51,001] (cue 37) route it to a human only when the risk is, you know, justifying that.

[00:02:51,041 --> 00:02:57,321] (cue 38) But this is, as I said before, this is not enough for large regulated enterprise

[00:02:57,361 --> 00:03:02,661] (cue 39) organizations, right? Because even when we auto-merge in this scenario,

[00:03:02,761 --> 00:03:08,281] (cue 40) then there are of questions about which agentic software factory created that

[00:03:08,321 --> 00:03:14,581] (cue 41) change. And do we trust it? There is no trust yet, just based on a risk classifier.

[00:03:15,721 --> 00:03:19,921] (cue 42) Every single time I talk with large enterprises going through this journey,

[00:03:19,961 --> 00:03:24,881] (cue 43) there are four elements that come up in conversations all the time about what's

[00:03:24,921 --> 00:03:28,001] (cue 44) needed. The first is verification.

[00:03:28,041 --> 00:03:34,321] (cue 45) How can we be sure that what the software factory produced as code meets the intent

[00:03:34,361 --> 00:03:40,921] (cue 46) we have? And that's it? Well, the second one is about, hey, did that code

[00:03:40,961 --> 00:03:46,081] (cue 47) change? Was that code change authored following our standards,

[00:03:46,121 --> 00:03:48,161] (cue 48) our platform engineering standards?

[00:03:48,201 --> 00:03:54,921] (cue 49) Our compliance frameworks? Did it did it follow our coding styles and so on?

[00:03:54,961 --> 00:03:58,261] (cue 50) Right. Which are often already in place.

[00:03:58,801 --> 00:04:00,961] (cue 51) The third one is traceability.

[00:04:01,001 --> 00:04:06,161] (cue 52) Like, you know, if I have a code change that I have merged that came from a software

[00:04:06,201 --> 00:04:10,681] (cue 53) factory that used, I don't know how many skills, how can I trace that back if

[00:04:10,721 --> 00:04:14,601] (cue 54) something goes wrong in production, how can I trace that back to the skill that I

[00:04:14,641 --> 00:04:20,841] (cue 55) need to improve? Or the factory that was compromised and and the last one is

[00:04:20,881 --> 00:04:25,641] (cue 56) security, and it's a little bit related to the, to the rest,

[00:04:25,681 --> 00:04:30,861] (cue 57) which is, you know, agentic software factories become a new surface of attack,

[00:04:30,961 --> 00:04:34,841] (cue 58) and it becomes a very lucrative and a very interesting surface of attack,

[00:04:34,881 --> 00:04:38,881] (cue 59) because now these software factories are going to generate all the code,

[00:04:38,921 --> 00:04:41,981] (cue 60) right? They will remove all humans from coding.

[00:04:42,561 --> 00:04:45,741] (cue 61) So the security aspect is super important.

[00:04:46,441 --> 00:04:52,321] (cue 62) Now I want us to think about how we build trust for classic software before the AI

[00:04:52,361 --> 00:04:58,051] (cue 63) era. And how did we build that, technically speaking, right.

[00:04:58,481 --> 00:05:02,161] (cue 64) To me, software runs on a fabric of trust.

[00:05:02,201 --> 00:05:04,561] (cue 65) Kind of, if I can use that metaphor.

[00:05:04,601 --> 00:05:09,081] (cue 66) There are several elements that brought us to where we are today,

[00:05:09,121 --> 00:05:13,961] (cue 67) where software runs every a lot of mission critical applications.

[00:05:14,001 --> 00:05:16,121] (cue 68) It all starts with versioning.

[00:05:16,161 --> 00:05:20,881] (cue 69) It starts with versioning because versioning is about giving an identity to software

[00:05:20,921 --> 00:05:23,801] (cue 70) artifacts. The fingerprint. Sure.

[00:05:23,841 --> 00:05:26,961] (cue 71) We see we see version 1.0, 2.0.

[00:05:27,001 --> 00:05:31,541] (cue 72) But behind that we have a digest and we have a fingerprint.

[00:05:31,881 --> 00:05:38,641] (cue 73) Once we have that, we can start creating the rest of the elements that compose this

[00:05:38,681 --> 00:05:43,641] (cue 74) fabric of trust like inventories, provenance, the evidence like.

[00:05:43,681 --> 00:05:49,881] (cue 75) And we have created a lot of standards so far to standardize this and speak the same

[00:05:49,921 --> 00:05:54,721] (cue 76) language. When we check, what should we run and what do we trust?

[00:05:54,761 --> 00:05:57,801] (cue 77) What software do we trust to be run in our environments?

[00:05:57,841 --> 00:06:02,621] (cue 78) Right? Like SLSA, CycloneDX, in-toto, and so on.

[00:06:03,801 --> 00:06:08,661] (cue 79) We invented signatures. And finally, ultimately with all of the above,

[00:06:08,761 --> 00:06:15,441] (cue 80) we could create admission. Admission checks, basically a policy that's unique to

[00:06:15,481 --> 00:06:19,801] (cue 81) each enterprise, which says, all right, I'm going to check the inventory of the bill

[00:06:19,841 --> 00:06:21,481] (cue 82) of materials of this software.

[00:06:21,521 --> 00:06:25,641] (cue 83) I'm going to check which organization created it and signed it.

[00:06:25,681 --> 00:06:30,801] (cue 84) Right. And based on all those things, I'm going to validate that this software can

[00:06:30,841 --> 00:06:37,168] (cue 85) run in my bank. All right. And so the question is when an agentic software factory

[00:06:37,208 --> 00:06:40,481] (cue 86) has generated software, where do we leave all of this.

[00:06:40,521 --> 00:06:44,041] (cue 87) What is the fabric of trust. Right.

[00:06:44,081 --> 00:06:47,821] (cue 88) I don't think we have a lot about that today yet.

[00:06:49,281 --> 00:06:53,761] (cue 89) So we need to build that and to build that.

[00:06:53,801 --> 00:06:59,761] (cue 90) I like to think about a factory as an agent program as software.

[00:06:59,801 --> 00:07:06,921] (cue 91) Right. Because we saw yesterday, for example, how Tessl, with the loops concept.

[00:07:06,961 --> 00:07:11,641] (cue 92) And some other vendors like ourselves with GitHub automations,

[00:07:11,681 --> 00:07:16,241] (cue 93) we call them automations. At the end of the day, it's a bunch of skills,

[00:07:16,281 --> 00:07:21,401] (cue 94) a bunch of prompts running a process that we have written in natural language.

[00:07:21,441 --> 00:07:26,781] (cue 95) Right? And a factory is a collection of those that are chained together.

[00:07:27,041 --> 00:07:31,261] (cue 96) That looks to me like a program, like a software program, chaining functions.

[00:07:31,801 --> 00:07:36,761] (cue 97) And so if it's a program, we should version it because it's the first thread of the

[00:07:36,801 --> 00:07:37,981] (cue 98) trust fabric.

[00:07:39,961 --> 00:07:45,441] (cue 99) The other problem we have to solve today is that in the proto factories we're

[00:07:45,481 --> 00:07:49,801] (cue 100) starting to create, we are gluing together a lot of loose parts.

[00:07:49,841 --> 00:07:56,401] (cue 101) Right. We saw skills, prompts, scripts, maybe some proxies and elements on the

[00:07:56,441 --> 00:08:01,081] (cue 102) networking layer, on the sandbox layer, environment layer, infrastructure layer.

[00:08:01,121 --> 00:08:03,421] (cue 103) It's a lot of loose parts right.

[00:08:04,041 --> 00:08:08,721] (cue 104) So we have all those loose parts with which we can compose a factory and build that

[00:08:08,761 --> 00:08:14,241] (cue 105) ourselves. And the other alternative is just go with a vendor.

[00:08:14,281 --> 00:08:18,841] (cue 106) Right. Which are starting now to propose a way to create factories.

[00:08:18,881 --> 00:08:21,144] (cue 107) Right. In the case of GitHub it's automations.

[00:08:21,184 --> 00:08:23,464] (cue 108) In the case of Tessl, it's loops.

[00:08:23,504 --> 00:08:27,481] (cue 109) In the case of Codex, no, of Cursor, it's also automations, right.

[00:08:27,521 --> 00:08:30,161] (cue 110) But that's getting us locked in right.

[00:08:30,201 --> 00:08:34,481] (cue 111) It's a nice UI, UX and so on. But once we build our factory there,

[00:08:34,521 --> 00:08:38,251] (cue 112) it's very hard to take it somewhere else and run it in another harness one day.

[00:08:38,291 --> 00:08:45,201] (cue 113) Right. So what's missing in between, in my opinion, is a single unique

[00:08:45,241 --> 00:08:52,081] (cue 114) way. Standard way to define the smallest element that defines a unit of work,

[00:08:52,121 --> 00:08:53,741] (cue 115) of agent work.

[00:08:55,161 --> 00:08:56,781] (cue 116) What do I mean with this?

[00:08:58,561 --> 00:09:03,281] (cue 117) This metaphor, I think, is going to make us converge.

[00:09:03,321 --> 00:09:06,561] (cue 118) So I call it agent contracts, right?

[00:09:06,601 --> 00:09:08,641] (cue 119) I don't call it automations. I don't call it loops.

[00:09:08,681 --> 00:09:14,561] (cue 120) I call it agent contracts in the same way that when we sign up for a job with a

[00:09:14,601 --> 00:09:21,521] (cue 121) company, we sign a contract. I do think that what can help us standardize and make

[00:09:21,561 --> 00:09:26,341] (cue 122) software factories portable are agent contracts.

[00:09:26,401 --> 00:09:33,121] (cue 123) As a simple markdown file where we define the assignment or we admit we basically

[00:09:33,161 --> 00:09:36,801] (cue 124) has an input external input the assignment.

[00:09:36,841 --> 00:09:43,541] (cue 125) The assignment can change when they when they the agent can be asked to perform one

[00:09:43,641 --> 00:09:47,161] (cue 126) one software change in one piece of software the next day another one.

[00:09:47,201 --> 00:09:53,281] (cue 127) But what doesn't change is the role, the role in which the agent signs itself for in

[00:09:53,321 --> 00:09:58,361] (cue 128) this contract, the dependencies, what's made available for the agent to perform this

[00:09:58,401 --> 00:10:04,161] (cue 129) job, and the budget. Like how much are we happy to pay for right?

[00:10:04,201 --> 00:10:09,881] (cue 130) For this job to get done? It resembles a lot the kind of work contracts that we sign

[00:10:09,921 --> 00:10:15,161] (cue 131) in real life, right? And what we get back are the results of the checks,

[00:10:15,201 --> 00:10:21,897] (cue 132) the verification and the deliverables, like software changes or change diffs or

[00:10:21,937 --> 00:10:28,881] (cue 133) pull requests. Right. So once we are able to define

[00:10:28,921 --> 00:10:33,881] (cue 134) agent contracts in a simple markdown file with front matter YAML or,

[00:10:33,921 --> 00:10:38,301] (cue 135) you know, that's a possible format for it, then we can version it.

[00:10:39,321 --> 00:10:44,121] (cue 136) This contract can link the skills and everything that's required for the agent to do

[00:10:44,161 --> 00:10:49,701] (cue 137) the job, and it becomes a program an agentic program.

[00:10:50,201 --> 00:10:54,281] (cue 138) It becomes a program that is versioned, that allows composition.

[00:10:54,321 --> 00:10:57,601] (cue 139) We can start composing contracts between themselves.

[00:10:57,641 --> 00:11:03,041] (cue 140) It allows portability if more harnesses and vendors accept it,

[00:11:03,081 --> 00:11:08,501] (cue 141) and it's identifiable, and it allows us to start building the fabric of trust.

[00:11:08,681 --> 00:11:13,921] (cue 142) All right. So a factory can be thought of as a graph of contracts.

[00:11:13,961 --> 00:11:18,881] (cue 143) Like if we have an agent contract, that's for planning another one for a spec,

[00:11:18,921 --> 00:11:22,281] (cue 144) then another one for building and implementing the actual code,

[00:11:22,321 --> 00:11:24,581] (cue 145) for documentation, for reviewing.

[00:11:24,801 --> 00:11:30,061] (cue 146) For a lot of different tasks, we chain them together and we have our factory.

[00:11:30,441 --> 00:11:34,361] (cue 147) We give it a version, we get the fingerprint and we have the identity of the

[00:11:34,401 --> 00:11:36,341] (cue 148) factory. Right.

[00:11:37,641 --> 00:11:42,741] (cue 149) And so if factories are programs, then once we have that definition,

[00:11:42,801 --> 00:11:49,601] (cue 150) what runs it. Right. And so I want to get my I want to get it to

[00:11:49,641 --> 00:11:56,701] (cue 151) run right now. So let's let's imagine we have basically the

[00:11:57,041 --> 00:11:59,921] (cue 152) the pull request example that I showed at the beginning.

[00:11:59,961 --> 00:12:05,881] (cue 153) The intent here defined in markdown hold large wires to new right.

[00:12:05,921 --> 00:12:11,761] (cue 154) This is intent. It can come in the form of a of an issue of a markdown file,

[00:12:11,801 --> 00:12:14,761] (cue 155) or a message from a colleague on Slack or whatever it is.

[00:12:14,801 --> 00:12:18,721] (cue 156) Right. I have defined here, I have defined it here in a markdown file.

[00:12:18,761 --> 00:12:25,561] (cue 157) So here we have that right. And this is our core banking software

[00:12:25,601 --> 00:12:29,281] (cue 158) Contoso Payments whatever. Okay great.

[00:12:29,321 --> 00:12:33,641] (cue 159) And then we have our factory. This is our these are all public repositories.

[00:12:33,681 --> 00:12:35,761] (cue 160) You will get access to that later.

[00:12:35,801 --> 00:12:40,381] (cue 161) And so in our factory we have a contract folder.

[00:12:40,641 --> 00:12:45,361] (cue 162) And this contract folders we have a bunch of contracts like from the graph that

[00:12:45,401 --> 00:12:52,201] (cue 163) you've seen before. And for example the build contract which specifies hey as input

[00:12:52,241 --> 00:12:57,561] (cue 164) I need the intent, the request, then the specification from the previous contract I

[00:12:57,601 --> 00:13:03,001] (cue 165) will produce a diff and I will import this specific skill.

[00:13:03,041 --> 00:13:05,361] (cue 166) Right. And I have this budget and so on.

[00:13:05,401 --> 00:13:10,561] (cue 167) And this is the the role the role of the agent contract.

[00:13:10,601 --> 00:13:15,241] (cue 168) All right. So those are those are our program files.

[00:13:15,281 --> 00:13:21,761] (cue 169) And then we have this APM YAML file which can be thought of as a package.json file

[00:13:21,801 --> 00:13:24,881] (cue 170) in the npm world. This is from agent package manager.

[00:13:24,921 --> 00:13:26,681] (cue 171) And it's defining our dependencies.

[00:13:26,721 --> 00:13:31,181] (cue 172) Like what are the skills that are required for me to run this factory?

[00:13:31,241 --> 00:13:36,721] (cue 173) And once we have this manifest here defining the dependencies and they get

[00:13:36,761 --> 00:13:43,481] (cue 174) installed, then our contracts can link them, import them as modules as we do in a in

[00:13:43,521 --> 00:13:46,241] (cue 175) a usual programming languages.

[00:13:46,281 --> 00:13:52,841] (cue 176) Right. And so with that I'll use apmx.

[00:13:52,881 --> 00:13:56,961] (cue 177) And you can see the analogy here to the npm world or UV or whatever.

[00:13:57,001 --> 00:14:02,881] (cue 178) Right. We have npm, npx, we have uv, uvx, you can see the analogies and I will use

[00:14:02,921 --> 00:14:06,961] (cue 179) that to run the factory and I will run it.

[00:14:07,001 --> 00:14:09,601] (cue 180) And I hope you can see that in two terminals.

[00:14:09,641 --> 00:14:14,241] (cue 181) One is going to basically run the factory on copilot as a harness.

[00:14:14,281 --> 00:14:16,221] (cue 182) I'm going to launch that right now.

[00:14:16,481 --> 00:14:19,861] (cue 183) And on the right hand side it's going to run it on OpenCode.

[00:14:20,361 --> 00:14:24,901] (cue 184) All right. And you can see that the first thing that's happening is that agent

[00:14:24,941 --> 00:14:29,241] (cue 185) package manager, which is akin to npm, is reading the manifest,

[00:14:29,281 --> 00:14:31,841] (cue 186) installing the dependencies, locking them in a lock file.

[00:14:31,881 --> 00:14:34,081] (cue 187) And then the factory can run as a program.

[00:14:34,121 --> 00:14:37,321] (cue 188) And we can see here the chaining of the contracts.

[00:14:37,361 --> 00:14:42,121] (cue 189) The first one is called planning being run on the harness of choice.

[00:14:42,161 --> 00:14:48,801] (cue 190) So these are the of how we can have a portable open format that can eventually run

[00:14:48,841 --> 00:14:52,641] (cue 191) on any harness. Right. I will leave that running.

[00:14:52,681 --> 00:14:53,481] (cue 192) Okay.

[00:15:36,360 --> 00:15:39,240] (cue 204) And let's come back to, to the presentation.

[00:15:39,280 --> 00:15:43,920] (cue 205) Okay. So APM I have introduced that briefly.

[00:15:43,960 --> 00:15:45,840] (cue 206) It's a package manager for agents.

[00:15:45,880 --> 00:15:50,000] (cue 207) It's an open source project. I started that independently.

[00:15:50,040 --> 00:15:55,320] (cue 208) And then as I am employed by Microsoft, I transfer it to the Microsoft Open Source

[00:15:55,360 --> 00:16:00,720] (cue 209) organization. All right. The intent here is to move it to a neutral foundation

[00:16:00,760 --> 00:16:03,520] (cue 210) working on that. And it is pretty simple.

[00:16:03,560 --> 00:16:06,040] (cue 211) We have a manifest with the dependencies.

[00:16:06,080 --> 00:16:09,280] (cue 212) We have a lock file. That's when we install.

[00:16:09,320 --> 00:16:14,240] (cue 213) And then we can create a policy in the enterprise which says what are the

[00:16:14,280 --> 00:16:18,240] (cue 214) dependencies we want to allow and what are the dependencies that we want to deny.

[00:16:18,280 --> 00:16:20,580] (cue 215) And many other cool things. All right.

[00:16:20,840 --> 00:16:26,760] (cue 216) And so APM is about running these modules as programs.

[00:16:26,800 --> 00:16:28,360] (cue 217) They contracts as programs. Right.

[00:16:28,400 --> 00:16:32,800] (cue 218) Which depend on skills, MCPs, hooks and whatever it is.

[00:16:32,840 --> 00:16:36,120] (cue 219) Right. So we can run it on any coding agent.

[00:16:36,160 --> 00:16:39,760] (cue 220) And as a result we are going to get our pull request, our diff.

[00:16:39,800 --> 00:16:41,876] (cue 221) But we are also going to get a receipt.

[00:16:41,916 --> 00:16:44,200] (cue 222) We are going to get a receipt.

[00:16:44,240 --> 00:16:47,280] (cue 223) And that's the second important element here.

[00:16:47,320 --> 00:16:51,700] (cue 224) One is the contracts and the other is the receipt of the run.

[00:16:52,920 --> 00:16:54,680] (cue 225) And so why is the receipt important.

[00:16:54,720 --> 00:16:57,760] (cue 226) All right let's go back to to how this works.

[00:16:57,800 --> 00:17:02,800] (cue 227) We get intent in. We have the factory as a as a chain of contracts or loops or

[00:17:02,840 --> 00:17:04,520] (cue 228) automations or whatever it is.

[00:17:04,560 --> 00:17:08,748] (cue 229) Right. It generates the diff, of course, but also the receipt.

[00:17:08,788 --> 00:17:15,560] (cue 230) And the receipt is signed. The receipt is signed by whichever machine which run our

[00:17:15,600 --> 00:17:20,880] (cue 231) factory, which we want, we choose to trust as the runner of our factory.

[00:17:20,920 --> 00:17:24,920] (cue 232) All right then we have our rules, our policies which define,

[00:17:24,960 --> 00:17:26,840] (cue 233) hey, which factories do we trust?

[00:17:26,880 --> 00:17:30,040] (cue 234) Which skills do we allow? In which ones do we deny?

[00:17:30,080 --> 00:17:32,280] (cue 235) Oh, we heard about that skill that was compromised.

[00:17:32,320 --> 00:17:33,920] (cue 236) We're going to deny it there, right?

[00:17:33,960 --> 00:17:37,480] (cue 237) It's our set of rules. And then we have a gate.

[00:17:37,520 --> 00:17:42,000] (cue 238) And that's typically implemented in CI/CD in the pull request targeting main.

[00:17:42,040 --> 00:17:45,640] (cue 239) Right. Those are the checks that can run in CI/CD.

[00:17:45,680 --> 00:17:51,032] (cue 240) And I've chosen to implement that as a demonstrator with a command,

[00:17:51,072 --> 00:17:57,920] (cue 241) apmx audit. It's just going to read the receipt read the policy check whether there

[00:17:57,960 --> 00:18:03,600] (cue 242) is you know, there is alignment between those and block the the pull request or auto

[00:18:03,640 --> 00:18:04,460] (cue 243) merge it.

[00:18:06,000 --> 00:18:08,140] (cue 244) All right. So let's look at the receipt.

[00:18:08,280 --> 00:18:14,600] (cue 245) The receipt is based on open standards that we all know about in the software supply

[00:18:14,640 --> 00:18:20,480] (cue 246) chain world. So if we look at the first block that's based on a provenance and it

[00:18:20,520 --> 00:18:25,840] (cue 247) gives us traceability, it allows us to understand that the output,

[00:18:25,880 --> 00:18:32,611] (cue 248) the diff that was generated comes from this factory because they are chained

[00:18:32,651 --> 00:18:36,120] (cue 249) through their fingerprints, through the digests.

[00:18:36,160 --> 00:18:41,640] (cue 250) Right. The versions. Right. Then we have the results of the checks,

[00:18:41,680 --> 00:18:45,880] (cue 251) the checks that have been defined in each contract and are immutable by the agents.

[00:18:45,920 --> 00:18:47,960] (cue 252) The agents cannot change those checks.

[00:18:48,000 --> 00:18:51,360] (cue 253) Remember that agents cannot be trusted, right?

[00:18:51,400 --> 00:18:53,520] (cue 254) They can change the rules of the game.

[00:18:53,560 --> 00:18:59,320] (cue 255) So we need to also cryptographically chain the results of the checks and the checks

[00:18:59,360 --> 00:19:01,320] (cue 256) themselves in the respect and the evidence.

[00:19:01,360 --> 00:19:03,500] (cue 257) All right. And that's based on in-toto.

[00:19:04,000 --> 00:19:06,640] (cue 258) And then we have our agent bill of materials.

[00:19:06,680 --> 00:19:08,680] (cue 259) It's akin to the software bill of materials.

[00:19:08,720 --> 00:19:11,520] (cue 260) It's about what's the composition of the factory.

[00:19:11,560 --> 00:19:15,320] (cue 261) Right. And finally we sign it.

[00:19:15,360 --> 00:19:20,640] (cue 262) We sign it with a trusted signer that cannot be the same machine that run the

[00:19:20,680 --> 00:19:23,000] (cue 263) factory because we cannot trust the agents.

[00:19:23,040 --> 00:19:25,200] (cue 264) Right. So it has to be something else.

[00:19:25,240 --> 00:19:28,060] (cue 265) And that gives us security, right?

[00:19:28,760 --> 00:19:34,040] (cue 266) And ultimately, it's this receipt which allows us to build that fabric of trust.

[00:19:34,080 --> 00:19:38,280] (cue 267) Right. So we get the identity as the factory version of fingerprint.

[00:19:38,320 --> 00:19:43,160] (cue 268) We get the inventory as the agent bill of materials out of the APM lock file.

[00:19:43,200 --> 00:19:49,089] (cue 269) Right. We get the provenance with a we get the evidence with in-toto,

[00:19:49,129 --> 00:19:53,150] (cue 270) thanks to the check results and the checks themselves chained to the receipt,

[00:19:53,190 --> 00:19:59,320] (cue 271) chained to the output. And we get, you know, we get attested provenance with the

[00:19:59,360 --> 00:20:06,160] (cue 272) signature. And ultimately we can create a gate to automate based

[00:20:06,200 --> 00:20:10,220] (cue 273) on actual evidence, not just risk classification.

[00:20:10,480 --> 00:20:15,240] (cue 274) Right. With a simple command, for example APM audit.

[00:20:15,280 --> 00:20:19,040] (cue 275) My goal here is not to position this tool or that tool, but what's needed,

[00:20:19,080 --> 00:20:22,380] (cue 276) what we need as an industry to create trust.

[00:20:22,440 --> 00:20:23,240] (cue 277) So

[00:20:24,560 --> 00:20:27,880] (cue 278) what happens then? All right. So the model is pretty simple.

[00:20:27,920 --> 00:20:29,120] (cue 279) We have the receipt. We have the gate.

[00:20:29,160 --> 00:20:35,540] (cue 280) We have the rules. We have that command APM audit that we can run

[00:20:36,240 --> 00:20:40,320] (cue 281) on pull requests as a check which is going to look at the evidence,

[00:20:40,360 --> 00:20:42,320] (cue 282) look at the diff as the output.

[00:20:42,360 --> 00:20:45,760] (cue 283) Make sure that there is the integrity, the provenance, the checks passed,

[00:20:45,800 --> 00:20:50,400] (cue 284) that there was no tampering, that we trust the signer, that the policy that we have

[00:20:50,440 --> 00:20:55,600] (cue 285) defined is respected. And then we're going to have a result either valid or invalid.

[00:20:55,640 --> 00:21:01,360] (cue 286) Right. And that's the verification of, you know, that fabric of trust that we have

[00:21:01,400 --> 00:21:04,560] (cue 287) chosen to to basically accept.

[00:21:04,600 --> 00:21:08,660] (cue 288) Right. All right. So I'm going to move faster here.

[00:21:09,800 --> 00:21:10,660] (cue 289) Okay.

[00:21:12,160 --> 00:21:18,480] (cue 290) What this allows us to do is to basically forgo human review,

[00:21:18,520 --> 00:21:24,280] (cue 291) forgo CI processes that are creating bottlenecks and auto merge,

[00:21:24,320 --> 00:21:28,880] (cue 292) but not just based on risk classification, on our personal confidence or very smart

[00:21:28,920 --> 00:21:34,560] (cue 293) engineers we have hired who say, yes, I trust my factory, but actual evidence that

[00:21:34,600 --> 00:21:40,320] (cue 294) we can trace back. And then when an auditor comes in, because we are regulated

[00:21:40,360 --> 00:21:44,840] (cue 295) organization like a bank or a pharma company, if something goes wrong,

[00:21:44,880 --> 00:21:51,640] (cue 296) we can point exactly at our controls that follow a certain regulatory framework.

[00:21:51,680 --> 00:21:56,400] (cue 297) We can point back to the skill or the factory that we decided to trust.

[00:21:56,440 --> 00:21:59,880] (cue 298) We can point back to what went wrong, and we can take action,

[00:21:59,920 --> 00:22:02,860] (cue 299) and we can place the responsibility where it has to be.

[00:22:04,440 --> 00:22:09,760] (cue 300) All right. So there are several questions that this approach help us answer.

[00:22:09,800 --> 00:22:12,780] (cue 301) Right. So for example production broke.

[00:22:13,240 --> 00:22:17,960] (cue 302) And so if production broke and everything in production was built by a factory.

[00:22:18,000 --> 00:22:22,800] (cue 303) Then the question is which factory is responsible and which step?

[00:22:22,840 --> 00:22:29,600] (cue 304) Precisely because we have this receipt that chains the outputs to the evidence.

[00:22:29,640 --> 00:22:35,120] (cue 305) Then we can trace back from the source code back to the receipt and back to the

[00:22:35,160 --> 00:22:39,560] (cue 306) factory, thanks to the versioning, which starts everything because it assigns an

[00:22:39,600 --> 00:22:40,660] (cue 307) identity.

[00:22:41,920 --> 00:22:46,640] (cue 308) Another question we can answer is hey, a skill in the open source world that we are

[00:22:46,680 --> 00:22:50,000] (cue 309) using was compromised. So who's exposed?

[00:22:50,040 --> 00:22:54,880] (cue 310) Just look at the agent bill of materials or every single receipt that you stored.

[00:22:54,920 --> 00:23:00,240] (cue 311) Right. And then find which factories are using those and patch it,

[00:23:00,280 --> 00:23:05,120] (cue 312) revoke it in the rules. The gate will start forbidding any output from those

[00:23:05,160 --> 00:23:08,620] (cue 313) factories from going to production until you fix that.

[00:23:08,920 --> 00:23:14,680] (cue 314) All right. Okay. So then next question the checks pass the checks that we have

[00:23:14,720 --> 00:23:18,560] (cue 315) defined in the contracts, which are exactly our our quality bar.

[00:23:18,600 --> 00:23:22,400] (cue 316) Now they passed but did the agent change them right.

[00:23:22,440 --> 00:23:26,560] (cue 317) We have seen agents gaming the checks consistently.

[00:23:26,600 --> 00:23:29,160] (cue 318) So hey checks lives in the contracts.

[00:23:29,200 --> 00:23:35,040] (cue 319) They don't. They are not expected to be created by the same agent who work in the

[00:23:35,080 --> 00:23:38,040] (cue 320) factory. So they cannot be changed.

[00:23:38,080 --> 00:23:41,360] (cue 321) They are cryptographically tied into the receipt.

[00:23:41,400 --> 00:23:47,640] (cue 322) So if somebody tampers those, you are going to find out and they are enforced in the

[00:23:47,680 --> 00:23:50,120] (cue 323) gate with the audit command. Right?

[00:23:50,160 --> 00:23:53,460] (cue 324) So it's not possible to tamper those.

[00:23:54,000 --> 00:23:58,480] (cue 325) All right. And so let's imagine hey we want to add new checks because of course the

[00:23:58,520 --> 00:24:03,660] (cue 326) checks at the end become here our bar for automate for trust right.

[00:24:04,040 --> 00:24:07,020] (cue 327) So how can we ensure that they become mandatory.

[00:24:07,480 --> 00:24:11,120] (cue 328) Well if we create new checks these changes the factory.

[00:24:11,160 --> 00:24:14,940] (cue 329) And so we can basically release a new factory version.

[00:24:15,240 --> 00:24:18,400] (cue 330) Right. If we change the checks which are tied in the contracts,

[00:24:18,440 --> 00:24:20,680] (cue 331) then we have a new version of our factory.

[00:24:20,720 --> 00:24:25,320] (cue 332) If we create a new version of our factory, then we can go to our policy file and

[00:24:25,360 --> 00:24:32,120] (cue 333) say, now we are only going to accept diffs coming from factories

[00:24:32,160 --> 00:24:34,480] (cue 334) above version 2.1 for example.

[00:24:34,520 --> 00:24:41,340] (cue 335) Right. And so our apmx audit command, the gate will forbid any output coming from

[00:24:41,760 --> 00:24:45,620] (cue 336) older factory versions. All right.

[00:24:46,440 --> 00:24:49,920] (cue 337) And so well there is another question that we can answer.

[00:24:49,960 --> 00:24:52,880] (cue 338) You know, in the policy we can also define, hey, what are the skills that the

[00:24:52,920 --> 00:24:55,080] (cue 339) factories can use are allowed to use.

[00:24:55,120 --> 00:24:58,260] (cue 340) Again, we have that policy in the rules.

[00:24:59,300 --> 00:25:02,600] (cue 341) The dependencies are tied in the manifest in the lock file.

[00:25:02,640 --> 00:25:07,740] (cue 342) We can check with the audit command and we can get based on that.

[00:25:08,080 --> 00:25:13,340] (cue 343) Those are just some examples of what this approach can help us with.

[00:25:13,800 --> 00:25:20,240] (cue 344) So now if I go back to the factory runs, I can see on the copilot run

[00:25:20,280 --> 00:25:22,920] (cue 345) that the different contracts were run.

[00:25:22,960 --> 00:25:26,620] (cue 346) Planning a specification, build a review.

[00:25:27,080 --> 00:25:33,260] (cue 347) What allows the state to flow through the contracts is basically files,

[00:25:33,440 --> 00:25:34,920] (cue 348) right? So it's markdown files.

[00:25:34,960 --> 00:25:37,520] (cue 349) It's diffs. It's whatever files that go into disk.

[00:25:37,560 --> 00:25:41,280] (cue 350) So a contract generates files and under contract will consume those files.

[00:25:41,320 --> 00:25:48,000] (cue 351) Right. We see it completed and we can see the outputs it generated the receipt that

[00:25:48,040 --> 00:25:51,880] (cue 352) was generated in total in an in-toto file.

[00:25:51,920 --> 00:25:58,500] (cue 353) And I can run the audit command quickly to show hey

[00:25:58,600 --> 00:26:02,780] (cue 354) now eventually if we run this audit command which should run in CI/CD,

[00:26:04,400 --> 00:26:09,680] (cue 355) can we check that the integrity is there, that the provenance is the one that we

[00:26:09,720 --> 00:26:13,120] (cue 356) that we decided to trust that the checks ran, that they were valid,

[00:26:13,160 --> 00:26:16,540] (cue 357) etc.. So we can see that happening there locally.

[00:26:16,640 --> 00:26:19,920] (cue 358) Of course here we didn't sign the receipt.

[00:26:19,960 --> 00:26:26,440] (cue 359) So the check doesn't pass. Eventually what we need to do is to add

[00:26:26,480 --> 00:26:31,300] (cue 360) that add that command as a check to a pull request.

[00:26:31,360 --> 00:26:35,700] (cue 361) So I have here a pull request generated by the factory.

[00:26:36,880 --> 00:26:41,420] (cue 362) And as a check I have a single one, the audit command.

[00:26:41,640 --> 00:26:46,736] (cue 363) We should have also code scanning and stuff like that which runs the apmx audit

[00:26:46,776 --> 00:26:52,960] (cue 364) command. And the basically all the checks pass the receipt is signed.

[00:26:53,000 --> 00:26:56,920] (cue 365) It was signed by the environment which run the factory.

[00:26:56,960 --> 00:26:59,800] (cue 366) You know it can be CI/CD, it can be our Kubernetes cluster.

[00:26:59,840 --> 00:27:02,520] (cue 367) It can be wherever you decide to run your factories.

[00:27:02,560 --> 00:27:07,720] (cue 368) And that becomes a valid statement that allows us to say, okay,

[00:27:07,760 --> 00:27:09,800] (cue 369) can we auto-merge this now? Right.

[00:27:09,840 --> 00:27:13,580] (cue 370) And we have traceability and all those properties I spoke about.

[00:27:14,920 --> 00:27:21,460] (cue 371) All right. So I'm going to close by saying we we seem to be all converging

[00:27:21,520 --> 00:27:28,280] (cue 372) into something like some sort of definition of the smallest unit of what a factory

[00:27:28,320 --> 00:27:31,600] (cue 373) is. You know, Tessl was speaking yesterday about loops.

[00:27:31,640 --> 00:27:34,880] (cue 374) We have a lot of vendors speaking about automations.

[00:27:34,920 --> 00:27:36,780] (cue 375) I mean, speaking about contracts.

[00:27:37,880 --> 00:27:40,880] (cue 376) What I do think is that we need a shared format.

[00:27:40,920 --> 00:27:46,600] (cue 377) If we do want portability of of software factories, we need a shared format,

[00:27:46,640 --> 00:27:50,960] (cue 378) a shared standard, and we need not to reinvent the wheel.

[00:27:51,000 --> 00:27:55,660] (cue 379) We created a lot of this fabric of trust in the past for classic software.

[00:27:55,720 --> 00:28:00,280] (cue 380) Let's not reinvent the wheel and use the lessons learned to build the same fabric of

[00:28:00,320 --> 00:28:02,960] (cue 381) trust for agentic software factories.

[00:28:03,000 --> 00:28:05,740] (cue 382) And let's do that in the open, right?

[00:28:05,960 --> 00:28:10,160] (cue 383) So we need to define factories as code in an open format.

[00:28:10,200 --> 00:28:15,180] (cue 384) We need to version them because it all starts with versioning which is identity.

[00:28:15,240 --> 00:28:19,600] (cue 385) And we need then we can then get on evidence on actual evidence.

[00:28:19,640 --> 00:28:23,140] (cue 386) And and we get the traceability, the provenance, the portability.

[00:28:23,360 --> 00:28:26,020] (cue 387) We skip CI we skip peer reviews.

[00:28:26,360 --> 00:28:30,760] (cue 388) And I think this is how agentic software factories can scale to large regulated

[00:28:30,800 --> 00:28:36,360] (cue 389) enterprises. All right. So this is a proposition for an open format that I have

[00:28:36,400 --> 00:28:42,880] (cue 390) published. It's a draft I think this is the perfect conference to invite you to

[00:28:42,920 --> 00:28:44,940] (cue 391) participate in the conversation.

[00:28:45,040 --> 00:28:50,320] (cue 392) I hope that from there, we'll be able one day to say, hey, we're going to change the

[00:28:50,360 --> 00:28:55,260] (cue 393) payment systems in the bank and we can even auto merge.

[00:28:56,000 --> 00:28:57,480] (cue 394) All right. Thank you very much.

[00:28:57,520 --> 00:29:01,840] (cue 395) With that, I leave you with the QR code for all the content.

[00:29:01,880 --> 00:29:05,660] (cue 396) Thank you. And any questions. I think we have one minute.

[00:29:06,120 --> 00:29:08,140] (cue 397) All right. Thanks.

[00:29:10,290 --> 00:29:13,600] (cue 398) Thank you. Thank you. Talk to you about the contents of it.

[00:29:13,640 --> 00:29:15,680] (cue 399) Thank you. Daniel. That was an amazing session.

[00:29:15,720 --> 00:29:18,960] (cue 400) Thanks. You're going to be doing questions now or do you want to stage?

[00:29:19,000 --> 00:29:22,000] (cue 401) Are you have time? Then I can pick a couple.

[00:29:22,040 --> 00:29:24,120] (cue 402) Of quick questions you want to ask you now.

[00:29:24,160 --> 00:29:27,180] (cue 403) Oh, there we go. I'll come down.

[00:29:29,320 --> 00:29:31,660] (cue 404) Let me go down. I can do the exercise.

[00:29:33,240 --> 00:29:37,280] (cue 405) So as part of the checks that you're defining is part of these kind of these agents.

[00:29:37,320 --> 00:29:39,840] (cue 406) First off, I love the kind of like, OPA design of all of this.

[00:29:39,880 --> 00:29:42,100] (cue 407) I think that makes a ton of sense.

[00:29:42,160 --> 00:29:48,280] (cue 408) But in regards to the checks, are you kind of alluding to kind of them like almost

[00:29:48,320 --> 00:29:51,340] (cue 409) being a replacement more for like unit and integration tests?

[00:29:51,760 --> 00:29:57,360] (cue 410) In that or is it more a specific kind of like a volatile of the specific code that

[00:29:57,400 --> 00:30:01,050] (cue 411) you're producing as part of that individual agent execution?

[00:30:01,090 --> 00:30:06,080] (cue 412) Yeah, yeah. So the checks have to be deterministic.

[00:30:06,120 --> 00:30:10,720] (cue 413) In my view. The checks that you define in a contract ought to be deterministic

[00:30:10,760 --> 00:30:15,880] (cue 414) because that's that's basically the harness around the model that you can basically

[00:30:15,920 --> 00:30:22,880] (cue 415) trust that elevates trust. So a check could be hey what is if we have a contract for

[00:30:22,920 --> 00:30:27,360] (cue 416) writing specifications, then maybe there is a specific format in what you want to

[00:30:27,400 --> 00:30:29,960] (cue 417) make sure that the specifications are written right.

[00:30:30,000 --> 00:30:35,040] (cue 418) So that could be one check if you have a contract for performing pull review pull

[00:30:35,080 --> 00:30:37,640] (cue 419) request reviews, then maybe it's about the format.

[00:30:37,680 --> 00:30:44,400] (cue 420) Maybe if you have one for building diffs actual code, then maybe it's about

[00:30:44,440 --> 00:30:49,840] (cue 421) hey, you can attach this piece of code or you have to follow this style.

[00:30:49,880 --> 00:30:52,540] (cue 422) So linting and stuff like that, right?

[00:30:52,630 --> 00:30:53,430] (cue 423) Yeah.

[00:31:03,880 --> 00:31:06,390] (cue 424) Staging and so on. And so

[00:31:09,300 --> 00:31:12,240] (cue 425) to load on CI. Exactly. How that

[00:31:24,480 --> 00:31:29,000] (cue 426) emerged. The idea is that you skip this running the checks again in CI because you

[00:31:29,040 --> 00:31:31,040] (cue 427) have the evidence that they have run.

[00:31:31,080 --> 00:31:34,520] (cue 428) You have trusted evidence sign that they have run and they have passed.

[00:31:34,560 --> 00:31:39,520] (cue 429) So the idea is that you can you you don't need to run them again on CI centrally.

[00:31:39,560 --> 00:31:44,220] (cue 430) So shifting left testing as much as possible in a in a trusted way.

[00:31:44,970 --> 00:31:49,140] (cue 431) Yeah. You go one more time for sure.

[00:31:50,480 --> 00:31:54,440] (cue 432) Hello. Waterfall was bad idea for humans.

[00:31:54,480 --> 00:31:57,660] (cue 433) Why would waterfall be a good idea for a AI?

[00:31:58,160 --> 00:32:00,720] (cue 434) Sorry. Can can you? Because they held of echo.

[00:32:00,760 --> 00:32:06,000] (cue 435) I didn't hear you well. So waterfall was bad idea for humans?

[00:32:06,040 --> 00:32:10,340] (cue 436) Yeah. Why would waterfall be a good idea for AI?

[00:32:11,800 --> 00:32:17,120] (cue 437) So, you know, I think waterfall did very well in some situations.

[00:32:17,160 --> 00:32:18,680] (cue 438) It still does well in some situations.

[00:32:18,720 --> 00:32:21,160] (cue 439) And I agree with you that it's not a solution for everything.

[00:32:21,200 --> 00:32:24,580] (cue 440) I think your question comes from the graph that you've seen,

[00:32:24,680 --> 00:32:27,560] (cue 441) which seems to be a waterfall approach.

[00:32:27,600 --> 00:32:33,000] (cue 442) I do think we are going to see forms in which there's nothing to do with waterfall.

[00:32:33,040 --> 00:32:39,680] (cue 443) Even when you do a scrum and all this, you follow a certain approach to

[00:32:39,720 --> 00:32:44,720] (cue 444) something. We are speaking about automation here, so I agree with you that waterfall

[00:32:44,760 --> 00:32:45,720] (cue 445) is not the answer for everything.

[00:32:45,760 --> 00:32:47,620] (cue 446) And this was just one example.

[00:32:47,760 --> 00:32:52,800] (cue 447) I think we can very likely be able to model another sort of process in the form of

[00:32:52,840 --> 00:32:54,180] (cue 448) contracts as well.

skills

talk-meppiel-versioned-agent-contracts

tile.json