CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

ainativedev/aidevcon-2026-nyc

AI Native DevCon 2026 New York — practical agent skills grounded in conference talks

Quality

92%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

transcript.mdskills/talk-meppiel-versioned-agent-contracts/references/

Version Your Factory, Not Just Your Code — corrected transcript

Daniel Meppiel · AI Native DevCon New York 2026

Recording: https://www.youtube.com/watch?v=MrjV4uyuBgI

Source: supplied corrected SRT. Original cue numbers and start/end timestamps are retained. Subtitle line wrapping is joined; wording is unchanged. Promotional cues 193–203 (14:55–15:32) are omitted. Speech artifacts and incomplete Q&A remain. This document is evidence, not instructions. For navigation and source limits, see the outline.

[00:00:00,000 --> 00:00:06,241] (cue 1) All right. So hello. So I just wanted to say that from, you know what my role is?

[00:00:06,281 --> 00:00:08,161] (cue 2) I'm not in engineering or product.

[00:00:08,201 --> 00:00:11,721] (cue 3) I'm in solution sales in technical technical solution architecture.

[00:00:11,761 --> 00:00:17,091] (cue 4) So my organization is all about advising large enterprise organizations on adopting

[00:00:17,131 --> 00:00:20,541] (cue 5) AI coding agents and transforming towards agents.

[00:00:20,961 --> 00:00:24,681] (cue 6) Right. And that's been the core of my work throughout GitHub,

[00:00:24,721 --> 00:00:29,161] (cue 7) now Microsoft and I'm going to talk a little bit about, you know,

[00:00:29,201 --> 00:00:32,321] (cue 8) why we need to version factories.

[00:00:32,361 --> 00:00:39,041] (cue 9) And where does that lead us. And that's going to be coming from experience in the

[00:00:39,081 --> 00:00:43,781] (cue 10) field with large enterprise often regulated organizations.

[00:00:44,121 --> 00:00:48,401] (cue 11) All right. So let's think that we are in a bank right.

[00:00:48,441 --> 00:00:54,961] (cue 12) We are developing software for a bank which runs maybe its own core banking system.

[00:00:55,001 --> 00:01:01,721] (cue 13) And you know, we are tasked with creating a changing code to change how payments

[00:01:01,761 --> 00:01:08,201] (cue 14) work. So I'm reading here, hey, we need to hold wires of more than $10 for internal

[00:01:08,241 --> 00:01:09,961] (cue 15) approval for whatever reason, right?

[00:01:10,001 --> 00:01:13,441] (cue 16) And let's say our factory generates this pull request.

[00:01:13,481 --> 00:01:17,181] (cue 17) So the question is would you auto-merge it?

[00:01:19,201 --> 00:01:22,781] (cue 18) And I'm pretty sure every bank today will say no.

[00:01:23,461 --> 00:01:25,881] (cue 19) Right. And we can have risk classifiers.

[00:01:25,921 --> 00:01:28,881] (cue 20) We can have verifiers. We can have a lot of these things.

[00:01:28,921 --> 00:01:34,281] (cue 21) But the answer to that question will very likely be no for some time.

[00:01:34,321 --> 00:01:40,561] (cue 22) And what I'm going to focus on is what do we need so that these can be automated one

[00:01:40,601 --> 00:01:44,721] (cue 23) way, one day confidently. All right.

[00:01:44,761 --> 00:01:50,001] (cue 24) So recapping, I think by now we all know what an agent software factory ideal or

[00:01:50,041 --> 00:01:55,641] (cue 25) concept is. We get the intent in, our agentic software factory runs an automated

[00:01:55,681 --> 00:02:00,741] (cue 26) agent process to basically transform this into code.

[00:02:00,881 --> 00:02:06,561] (cue 27) Code changes. Now the problem with that is that we create new bottlenecks.

[00:02:06,601 --> 00:02:09,781] (cue 28) And this has been repeated over and over and over yesterday.

[00:02:09,921 --> 00:02:11,961] (cue 29) One of those bottlenecks is CI.

[00:02:12,001 --> 00:02:16,721] (cue 30) We get so many requests, CI runs everything again and this becomes a bottleneck.

[00:02:16,761 --> 00:02:19,041] (cue 31) And the other one is us humans.

[00:02:19,081 --> 00:02:22,361] (cue 32) And we just heard in the keynote that we need to remove all humans.

[00:02:22,401 --> 00:02:28,121] (cue 33) Right. And if we don't do this agentic software factories cannot scale in large

[00:02:28,161 --> 00:02:33,121] (cue 34) organizations either. Right. So that's another piece of the problem that we need to

[00:02:33,161 --> 00:02:38,721] (cue 35) solve today. We seem to be converging on the idea of a risk classifier.

[00:02:38,761 --> 00:02:45,241] (cue 36) Right. Which is basically hey, we identify what the code change is changing and then

[00:02:45,281 --> 00:02:51,001] (cue 37) route it to a human only when the risk is, you know, justifying that.

[00:02:51,041 --> 00:02:57,321] (cue 38) But this is, as I said before, this is not enough for large regulated enterprise

[00:02:57,361 --> 00:03:02,661] (cue 39) organizations, right? Because even when we auto-merge in this scenario,

[00:03:02,761 --> 00:03:08,281] (cue 40) then there are of questions about which agentic software factory created that

[00:03:08,321 --> 00:03:14,581] (cue 41) change. And do we trust it? There is no trust yet, just based on a risk classifier.

[00:03:15,721 --> 00:03:19,921] (cue 42) Every single time I talk with large enterprises going through this journey,

[00:03:19,961 --> 00:03:24,881] (cue 43) there are four elements that come up in conversations all the time about what's

[00:03:24,921 --> 00:03:28,001] (cue 44) needed. The first is verification.

[00:03:28,041 --> 00:03:34,321] (cue 45) How can we be sure that what the software factory produced as code meets the intent

[00:03:34,361 --> 00:03:40,921] (cue 46) we have? And that's it? Well, the second one is about, hey, did that code

[00:03:40,961 --> 00:03:46,081] (cue 47) change? Was that code change authored following our standards,

[00:03:46,121 --> 00:03:48,161] (cue 48) our platform engineering standards?

[00:03:48,201 --> 00:03:54,921] (cue 49) Our compliance frameworks? Did it did it follow our coding styles and so on?

[00:03:54,961 --> 00:03:58,261] (cue 50) Right. Which are often already in place.

[00:03:58,801 --> 00:04:00,961] (cue 51) The third one is traceability.

[00:04:01,001 --> 00:04:06,161] (cue 52) Like, you know, if I have a code change that I have merged that came from a software

[00:04:06,201 --> 00:04:10,681] (cue 53) factory that used, I don't know how many skills, how can I trace that back if

[00:04:10,721 --> 00:04:14,601] (cue 54) something goes wrong in production, how can I trace that back to the skill that I

[00:04:14,641 --> 00:04:20,841] (cue 55) need to improve? Or the factory that was compromised and and the last one is

[00:04:20,881 --> 00:04:25,641] (cue 56) security, and it's a little bit related to the, to the rest,

[00:04:25,681 --> 00:04:30,861] (cue 57) which is, you know, agentic software factories become a new surface of attack,

[00:04:30,961 --> 00:04:34,841] (cue 58) and it becomes a very lucrative and a very interesting surface of attack,

[00:04:34,881 --> 00:04:38,881] (cue 59) because now these software factories are going to generate all the code,

[00:04:38,921 --> 00:04:41,981] (cue 60) right? They will remove all humans from coding.

[00:04:42,561 --> 00:04:45,741] (cue 61) So the security aspect is super important.

[00:04:46,441 --> 00:04:52,321] (cue 62) Now I want us to think about how we build trust for classic software before the AI

[00:04:52,361 --> 00:04:58,051] (cue 63) era. And how did we build that, technically speaking, right.

[00:04:58,481 --> 00:05:02,161] (cue 64) To me, software runs on a fabric of trust.

[00:05:02,201 --> 00:05:04,561] (cue 65) Kind of, if I can use that metaphor.

[00:05:04,601 --> 00:05:09,081] (cue 66) There are several elements that brought us to where we are today,

[00:05:09,121 --> 00:05:13,961] (cue 67) where software runs every a lot of mission critical applications.

[00:05:14,001 --> 00:05:16,121] (cue 68) It all starts with versioning.

[00:05:16,161 --> 00:05:20,881] (cue 69) It starts with versioning because versioning is about giving an identity to software

[00:05:20,921 --> 00:05:23,801] (cue 70) artifacts. The fingerprint. Sure.

[00:05:23,841 --> 00:05:26,961] (cue 71) We see we see version 1.0, 2.0.

[00:05:27,001 --> 00:05:31,541] (cue 72) But behind that we have a digest and we have a fingerprint.

[00:05:31,881 --> 00:05:38,641] (cue 73) Once we have that, we can start creating the rest of the elements that compose this

[00:05:38,681 --> 00:05:43,641] (cue 74) fabric of trust like inventories, provenance, the evidence like.

[00:05:43,681 --> 00:05:49,881] (cue 75) And we have created a lot of standards so far to standardize this and speak the same

[00:05:49,921 --> 00:05:54,721] (cue 76) language. When we check, what should we run and what do we trust?

[00:05:54,761 --> 00:05:57,801] (cue 77) What software do we trust to be run in our environments?

[00:05:57,841 --> 00:06:02,621] (cue 78) Right? Like SLSA, CycloneDX, in-toto, and so on.

[00:06:03,801 --> 00:06:08,661] (cue 79) We invented signatures. And finally, ultimately with all of the above,

[00:06:08,761 --> 00:06:15,441] (cue 80) we could create admission. Admission checks, basically a policy that's unique to

[00:06:15,481 --> 00:06:19,801] (cue 81) each enterprise, which says, all right, I'm going to check the inventory of the bill

[00:06:19,841 --> 00:06:21,481] (cue 82) of materials of this software.

[00:06:21,521 --> 00:06:25,641] (cue 83) I'm going to check which organization created it and signed it.

[00:06:25,681 --> 00:06:30,801] (cue 84) Right. And based on all those things, I'm going to validate that this software can

[00:06:30,841 --> 00:06:37,168] (cue 85) run in my bank. All right. And so the question is when an agentic software factory

[00:06:37,208 --> 00:06:40,481] (cue 86) has generated software, where do we leave all of this.

[00:06:40,521 --> 00:06:44,041] (cue 87) What is the fabric of trust. Right.

[00:06:44,081 --> 00:06:47,821] (cue 88) I don't think we have a lot about that today yet.

[00:06:49,281 --> 00:06:53,761] (cue 89) So we need to build that and to build that.

[00:06:53,801 --> 00:06:59,761] (cue 90) I like to think about a factory as an agent program as software.

[00:06:59,801 --> 00:07:06,921] (cue 91) Right. Because we saw yesterday, for example, how Tessl, with the loops concept.

[00:07:06,961 --> 00:07:11,641] (cue 92) And some other vendors like ourselves with GitHub automations,

[00:07:11,681 --> 00:07:16,241] (cue 93) we call them automations. At the end of the day, it's a bunch of skills,

[00:07:16,281 --> 00:07:21,401] (cue 94) a bunch of prompts running a process that we have written in natural language.

[00:07:21,441 --> 00:07:26,781] (cue 95) Right? And a factory is a collection of those that are chained together.

[00:07:27,041 --> 00:07:31,261] (cue 96) That looks to me like a program, like a software program, chaining functions.

[00:07:31,801 --> 00:07:36,761] (cue 97) And so if it's a program, we should version it because it's the first thread of the

[00:07:36,801 --> 00:07:37,981] (cue 98) trust fabric.

[00:07:39,961 --> 00:07:45,441] (cue 99) The other problem we have to solve today is that in the proto factories we're

[00:07:45,481 --> 00:07:49,801] (cue 100) starting to create, we are gluing together a lot of loose parts.

[00:07:49,841 --> 00:07:56,401] (cue 101) Right. We saw skills, prompts, scripts, maybe some proxies and elements on the

[00:07:56,441 --> 00:08:01,081] (cue 102) networking layer, on the sandbox layer, environment layer, infrastructure layer.

[00:08:01,121 --> 00:08:03,421] (cue 103) It's a lot of loose parts right.

[00:08:04,041 --> 00:08:08,721] (cue 104) So we have all those loose parts with which we can compose a factory and build that

[00:08:08,761 --> 00:08:14,241] (cue 105) ourselves. And the other alternative is just go with a vendor.

[00:08:14,281 --> 00:08:18,841] (cue 106) Right. Which are starting now to propose a way to create factories.

[00:08:18,881 --> 00:08:21,144] (cue 107) Right. In the case of GitHub it's automations.

[00:08:21,184 --> 00:08:23,464] (cue 108) In the case of Tessl, it's loops.

[00:08:23,504 --> 00:08:27,481] (cue 109) In the case of Codex, no, of Cursor, it's also automations, right.

[00:08:27,521 --> 00:08:30,161] (cue 110) But that's getting us locked in right.

[00:08:30,201 --> 00:08:34,481] (cue 111) It's a nice UI, UX and so on. But once we build our factory there,

[00:08:34,521 --> 00:08:38,251] (cue 112) it's very hard to take it somewhere else and run it in another harness one day.

[00:08:38,291 --> 00:08:45,201] (cue 113) Right. So what's missing in between, in my opinion, is a single unique

[00:08:45,241 --> 00:08:52,081] (cue 114) way. Standard way to define the smallest element that defines a unit of work,

[00:08:52,121 --> 00:08:53,741] (cue 115) of agent work.

[00:08:55,161 --> 00:08:56,781] (cue 116) What do I mean with this?

[00:08:58,561 --> 00:09:03,281] (cue 117) This metaphor, I think, is going to make us converge.

[00:09:03,321 --> 00:09:06,561] (cue 118) So I call it agent contracts, right?

[00:09:06,601 --> 00:09:08,641] (cue 119) I don't call it automations. I don't call it loops.

[00:09:08,681 --> 00:09:14,561] (cue 120) I call it agent contracts in the same way that when we sign up for a job with a

[00:09:14,601 --> 00:09:21,521] (cue 121) company, we sign a contract. I do think that what can help us standardize and make

[00:09:21,561 --> 00:09:26,341] (cue 122) software factories portable are agent contracts.

[00:09:26,401 --> 00:09:33,121] (cue 123) As a simple markdown file where we define the assignment or we admit we basically

[00:09:33,161 --> 00:09:36,801] (cue 124) has an input external input the assignment.

[00:09:36,841 --> 00:09:43,541] (cue 125) The assignment can change when they when they the agent can be asked to perform one

[00:09:43,641 --> 00:09:47,161] (cue 126) one software change in one piece of software the next day another one.

[00:09:47,201 --> 00:09:53,281] (cue 127) But what doesn't change is the role, the role in which the agent signs itself for in

[00:09:53,321 --> 00:09:58,361] (cue 128) this contract, the dependencies, what's made available for the agent to perform this

[00:09:58,401 --> 00:10:04,161] (cue 129) job, and the budget. Like how much are we happy to pay for right?

[00:10:04,201 --> 00:10:09,881] (cue 130) For this job to get done? It resembles a lot the kind of work contracts that we sign

[00:10:09,921 --> 00:10:15,161] (cue 131) in real life, right? And what we get back are the results of the checks,

[00:10:15,201 --> 00:10:21,897] (cue 132) the verification and the deliverables, like software changes or change diffs or

[00:10:21,937 --> 00:10:28,881] (cue 133) pull requests. Right. So once we are able to define

[00:10:28,921 --> 00:10:33,881] (cue 134) agent contracts in a simple markdown file with front matter YAML or,

[00:10:33,921 --> 00:10:38,301] (cue 135) you know, that's a possible format for it, then we can version it.

[00:10:39,321 --> 00:10:44,121] (cue 136) This contract can link the skills and everything that's required for the agent to do

[00:10:44,161 --> 00:10:49,701] (cue 137) the job, and it becomes a program an agentic program.

[00:10:50,201 --> 00:10:54,281] (cue 138) It becomes a program that is versioned, that allows composition.

[00:10:54,321 --> 00:10:57,601] (cue 139) We can start composing contracts between themselves.

[00:10:57,641 --> 00:11:03,041] (cue 140) It allows portability if more harnesses and vendors accept it,

[00:11:03,081 --> 00:11:08,501] (cue 141) and it's identifiable, and it allows us to start building the fabric of trust.

[00:11:08,681 --> 00:11:13,921] (cue 142) All right. So a factory can be thought of as a graph of contracts.

[00:11:13,961 --> 00:11:18,881] (cue 143) Like if we have an agent contract, that's for planning another one for a spec,

[00:11:18,921 --> 00:11:22,281] (cue 144) then another one for building and implementing the actual code,

[00:11:22,321 --> 00:11:24,581] (cue 145) for documentation, for reviewing.

[00:11:24,801 --> 00:11:30,061] (cue 146) For a lot of different tasks, we chain them together and we have our factory.

[00:11:30,441 --> 00:11:34,361] (cue 147) We give it a version, we get the fingerprint and we have the identity of the

[00:11:34,401 --> 00:11:36,341] (cue 148) factory. Right.

[00:11:37,641 --> 00:11:42,741] (cue 149) And so if factories are programs, then once we have that definition,

[00:11:42,801 --> 00:11:49,601] (cue 150) what runs it. Right. And so I want to get my I want to get it to

[00:11:49,641 --> 00:11:56,701] (cue 151) run right now. So let's let's imagine we have basically the

[00:11:57,041 --> 00:11:59,921] (cue 152) the pull request example that I showed at the beginning.

[00:11:59,961 --> 00:12:05,881] (cue 153) The intent here defined in markdown hold large wires to new right.

[00:12:05,921 --> 00:12:11,761] (cue 154) This is intent. It can come in the form of a of an issue of a markdown file,

[00:12:11,801 --> 00:12:14,761] (cue 155) or a message from a colleague on Slack or whatever it is.

[00:12:14,801 --> 00:12:18,721] (cue 156) Right. I have defined here, I have defined it here in a markdown file.

[00:12:18,761 --> 00:12:25,561] (cue 157) So here we have that right. And this is our core banking software

[00:12:25,601 --> 00:12:29,281] (cue 158) Contoso Payments whatever. Okay great.

[00:12:29,321 --> 00:12:33,641] (cue 159) And then we have our factory. This is our these are all public repositories.

[00:12:33,681 --> 00:12:35,761] (cue 160) You will get access to that later.

[00:12:35,801 --> 00:12:40,381] (cue 161) And so in our factory we have a contract folder.

[00:12:40,641 --> 00:12:45,361] (cue 162) And this contract folders we have a bunch of contracts like from the graph that

[00:12:45,401 --> 00:12:52,201] (cue 163) you've seen before. And for example the build contract which specifies hey as input

[00:12:52,241 --> 00:12:57,561] (cue 164) I need the intent, the request, then the specification from the previous contract I

[00:12:57,601 --> 00:13:03,001] (cue 165) will produce a diff and I will import this specific skill.

[00:13:03,041 --> 00:13:05,361] (cue 166) Right. And I have this budget and so on.

[00:13:05,401 --> 00:13:10,561] (cue 167) And this is the the role the role of the agent contract.

[00:13:10,601 --> 00:13:15,241] (cue 168) All right. So those are those are our program files.

[00:13:15,281 --> 00:13:21,761] (cue 169) And then we have this APM YAML file which can be thought of as a package.json file

[00:13:21,801 --> 00:13:24,881] (cue 170) in the npm world. This is from agent package manager.

[00:13:24,921 --> 00:13:26,681] (cue 171) And it's defining our dependencies.

[00:13:26,721 --> 00:13:31,181] (cue 172) Like what are the skills that are required for me to run this factory?

[00:13:31,241 --> 00:13:36,721] (cue 173) And once we have this manifest here defining the dependencies and they get

[00:13:36,761 --> 00:13:43,481] (cue 174) installed, then our contracts can link them, import them as modules as we do in a in

[00:13:43,521 --> 00:13:46,241] (cue 175) a usual programming languages.

[00:13:46,281 --> 00:13:52,841] (cue 176) Right. And so with that I'll use apmx.

[00:13:52,881 --> 00:13:56,961] (cue 177) And you can see the analogy here to the npm world or UV or whatever.

[00:13:57,001 --> 00:14:02,881] (cue 178) Right. We have npm, npx, we have uv, uvx, you can see the analogies and I will use

[00:14:02,921 --> 00:14:06,961] (cue 179) that to run the factory and I will run it.

[00:14:07,001 --> 00:14:09,601] (cue 180) And I hope you can see that in two terminals.

[00:14:09,641 --> 00:14:14,241] (cue 181) One is going to basically run the factory on copilot as a harness.

[00:14:14,281 --> 00:14:16,221] (cue 182) I'm going to launch that right now.

[00:14:16,481 --> 00:14:19,861] (cue 183) And on the right hand side it's going to run it on OpenCode.

[00:14:20,361 --> 00:14:24,901] (cue 184) All right. And you can see that the first thing that's happening is that agent

[00:14:24,941 --> 00:14:29,241] (cue 185) package manager, which is akin to npm, is reading the manifest,

[00:14:29,281 --> 00:14:31,841] (cue 186) installing the dependencies, locking them in a lock file.

[00:14:31,881 --> 00:14:34,081] (cue 187) And then the factory can run as a program.

[00:14:34,121 --> 00:14:37,321] (cue 188) And we can see here the chaining of the contracts.

[00:14:37,361 --> 00:14:42,121] (cue 189) The first one is called planning being run on the harness of choice.

[00:14:42,161 --> 00:14:48,801] (cue 190) So these are the of how we can have a portable open format that can eventually run

[00:14:48,841 --> 00:14:52,641] (cue 191) on any harness. Right. I will leave that running.

[00:14:52,681 --> 00:14:53,481] (cue 192) Okay.

[00:15:36,360 --> 00:15:39,240] (cue 204) And let's come back to, to the presentation.

[00:15:39,280 --> 00:15:43,920] (cue 205) Okay. So APM I have introduced that briefly.

[00:15:43,960 --> 00:15:45,840] (cue 206) It's a package manager for agents.

[00:15:45,880 --> 00:15:50,000] (cue 207) It's an open source project. I started that independently.

[00:15:50,040 --> 00:15:55,320] (cue 208) And then as I am employed by Microsoft, I transfer it to the Microsoft Open Source

[00:15:55,360 --> 00:16:00,720] (cue 209) organization. All right. The intent here is to move it to a neutral foundation

[00:16:00,760 --> 00:16:03,520] (cue 210) working on that. And it is pretty simple.

[00:16:03,560 --> 00:16:06,040] (cue 211) We have a manifest with the dependencies.

[00:16:06,080 --> 00:16:09,280] (cue 212) We have a lock file. That's when we install.

[00:16:09,320 --> 00:16:14,240] (cue 213) And then we can create a policy in the enterprise which says what are the

[00:16:14,280 --> 00:16:18,240] (cue 214) dependencies we want to allow and what are the dependencies that we want to deny.

[00:16:18,280 --> 00:16:20,580] (cue 215) And many other cool things. All right.

[00:16:20,840 --> 00:16:26,760] (cue 216) And so APM is about running these modules as programs.

[00:16:26,800 --> 00:16:28,360] (cue 217) They contracts as programs. Right.

[00:16:28,400 --> 00:16:32,800] (cue 218) Which depend on skills, MCPs, hooks and whatever it is.

[00:16:32,840 --> 00:16:36,120] (cue 219) Right. So we can run it on any coding agent.

[00:16:36,160 --> 00:16:39,760] (cue 220) And as a result we are going to get our pull request, our diff.

[00:16:39,800 --> 00:16:41,876] (cue 221) But we are also going to get a receipt.

[00:16:41,916 --> 00:16:44,200] (cue 222) We are going to get a receipt.

[00:16:44,240 --> 00:16:47,280] (cue 223) And that's the second important element here.

[00:16:47,320 --> 00:16:51,700] (cue 224) One is the contracts and the other is the receipt of the run.

[00:16:52,920 --> 00:16:54,680] (cue 225) And so why is the receipt important.

[00:16:54,720 --> 00:16:57,760] (cue 226) All right let's go back to to how this works.

[00:16:57,800 --> 00:17:02,800] (cue 227) We get intent in. We have the factory as a as a chain of contracts or loops or

[00:17:02,840 --> 00:17:04,520] (cue 228) automations or whatever it is.

[00:17:04,560 --> 00:17:08,748] (cue 229) Right. It generates the diff, of course, but also the receipt.

[00:17:08,788 --> 00:17:15,560] (cue 230) And the receipt is signed. The receipt is signed by whichever machine which run our

[00:17:15,600 --> 00:17:20,880] (cue 231) factory, which we want, we choose to trust as the runner of our factory.

[00:17:20,920 --> 00:17:24,920] (cue 232) All right then we have our rules, our policies which define,

[00:17:24,960 --> 00:17:26,840] (cue 233) hey, which factories do we trust?

[00:17:26,880 --> 00:17:30,040] (cue 234) Which skills do we allow? In which ones do we deny?

[00:17:30,080 --> 00:17:32,280] (cue 235) Oh, we heard about that skill that was compromised.

[00:17:32,320 --> 00:17:33,920] (cue 236) We're going to deny it there, right?

[00:17:33,960 --> 00:17:37,480] (cue 237) It's our set of rules. And then we have a gate.

[00:17:37,520 --> 00:17:42,000] (cue 238) And that's typically implemented in CI/CD in the pull request targeting main.

[00:17:42,040 --> 00:17:45,640] (cue 239) Right. Those are the checks that can run in CI/CD.

[00:17:45,680 --> 00:17:51,032] (cue 240) And I've chosen to implement that as a demonstrator with a command,

[00:17:51,072 --> 00:17:57,920] (cue 241) apmx audit. It's just going to read the receipt read the policy check whether there

[00:17:57,960 --> 00:18:03,600] (cue 242) is you know, there is alignment between those and block the the pull request or auto

[00:18:03,640 --> 00:18:04,460] (cue 243) merge it.

[00:18:06,000 --> 00:18:08,140] (cue 244) All right. So let's look at the receipt.

[00:18:08,280 --> 00:18:14,600] (cue 245) The receipt is based on open standards that we all know about in the software supply

[00:18:14,640 --> 00:18:20,480] (cue 246) chain world. So if we look at the first block that's based on a provenance and it

[00:18:20,520 --> 00:18:25,840] (cue 247) gives us traceability, it allows us to understand that the output,

[00:18:25,880 --> 00:18:32,611] (cue 248) the diff that was generated comes from this factory because they are chained

[00:18:32,651 --> 00:18:36,120] (cue 249) through their fingerprints, through the digests.

[00:18:36,160 --> 00:18:41,640] (cue 250) Right. The versions. Right. Then we have the results of the checks,

[00:18:41,680 --> 00:18:45,880] (cue 251) the checks that have been defined in each contract and are immutable by the agents.

[00:18:45,920 --> 00:18:47,960] (cue 252) The agents cannot change those checks.

[00:18:48,000 --> 00:18:51,360] (cue 253) Remember that agents cannot be trusted, right?

[00:18:51,400 --> 00:18:53,520] (cue 254) They can change the rules of the game.

[00:18:53,560 --> 00:18:59,320] (cue 255) So we need to also cryptographically chain the results of the checks and the checks

[00:18:59,360 --> 00:19:01,320] (cue 256) themselves in the respect and the evidence.

[00:19:01,360 --> 00:19:03,500] (cue 257) All right. And that's based on in-toto.

[00:19:04,000 --> 00:19:06,640] (cue 258) And then we have our agent bill of materials.

[00:19:06,680 --> 00:19:08,680] (cue 259) It's akin to the software bill of materials.

[00:19:08,720 --> 00:19:11,520] (cue 260) It's about what's the composition of the factory.

[00:19:11,560 --> 00:19:15,320] (cue 261) Right. And finally we sign it.

[00:19:15,360 --> 00:19:20,640] (cue 262) We sign it with a trusted signer that cannot be the same machine that run the

[00:19:20,680 --> 00:19:23,000] (cue 263) factory because we cannot trust the agents.

[00:19:23,040 --> 00:19:25,200] (cue 264) Right. So it has to be something else.

[00:19:25,240 --> 00:19:28,060] (cue 265) And that gives us security, right?

[00:19:28,760 --> 00:19:34,040] (cue 266) And ultimately, it's this receipt which allows us to build that fabric of trust.

[00:19:34,080 --> 00:19:38,280] (cue 267) Right. So we get the identity as the factory version of fingerprint.

[00:19:38,320 --> 00:19:43,160] (cue 268) We get the inventory as the agent bill of materials out of the APM lock file.

[00:19:43,200 --> 00:19:49,089] (cue 269) Right. We get the provenance with a we get the evidence with in-toto,

[00:19:49,129 --> 00:19:53,150] (cue 270) thanks to the check results and the checks themselves chained to the receipt,

[00:19:53,190 --> 00:19:59,320] (cue 271) chained to the output. And we get, you know, we get attested provenance with the

[00:19:59,360 --> 00:20:06,160] (cue 272) signature. And ultimately we can create a gate to automate based

[00:20:06,200 --> 00:20:10,220] (cue 273) on actual evidence, not just risk classification.

[00:20:10,480 --> 00:20:15,240] (cue 274) Right. With a simple command, for example APM audit.

[00:20:15,280 --> 00:20:19,040] (cue 275) My goal here is not to position this tool or that tool, but what's needed,

[00:20:19,080 --> 00:20:22,380] (cue 276) what we need as an industry to create trust.

[00:20:22,440 --> 00:20:23,240] (cue 277) So

[00:20:24,560 --> 00:20:27,880] (cue 278) what happens then? All right. So the model is pretty simple.

[00:20:27,920 --> 00:20:29,120] (cue 279) We have the receipt. We have the gate.

[00:20:29,160 --> 00:20:35,540] (cue 280) We have the rules. We have that command APM audit that we can run

[00:20:36,240 --> 00:20:40,320] (cue 281) on pull requests as a check which is going to look at the evidence,

[00:20:40,360 --> 00:20:42,320] (cue 282) look at the diff as the output.

[00:20:42,360 --> 00:20:45,760] (cue 283) Make sure that there is the integrity, the provenance, the checks passed,

[00:20:45,800 --> 00:20:50,400] (cue 284) that there was no tampering, that we trust the signer, that the policy that we have

[00:20:50,440 --> 00:20:55,600] (cue 285) defined is respected. And then we're going to have a result either valid or invalid.

[00:20:55,640 --> 00:21:01,360] (cue 286) Right. And that's the verification of, you know, that fabric of trust that we have

[00:21:01,400 --> 00:21:04,560] (cue 287) chosen to to basically accept.

[00:21:04,600 --> 00:21:08,660] (cue 288) Right. All right. So I'm going to move faster here.

[00:21:09,800 --> 00:21:10,660] (cue 289) Okay.

[00:21:12,160 --> 00:21:18,480] (cue 290) What this allows us to do is to basically forgo human review,

[00:21:18,520 --> 00:21:24,280] (cue 291) forgo CI processes that are creating bottlenecks and auto merge,

[00:21:24,320 --> 00:21:28,880] (cue 292) but not just based on risk classification, on our personal confidence or very smart

[00:21:28,920 --> 00:21:34,560] (cue 293) engineers we have hired who say, yes, I trust my factory, but actual evidence that

[00:21:34,600 --> 00:21:40,320] (cue 294) we can trace back. And then when an auditor comes in, because we are regulated

[00:21:40,360 --> 00:21:44,840] (cue 295) organization like a bank or a pharma company, if something goes wrong,

[00:21:44,880 --> 00:21:51,640] (cue 296) we can point exactly at our controls that follow a certain regulatory framework.

[00:21:51,680 --> 00:21:56,400] (cue 297) We can point back to the skill or the factory that we decided to trust.

[00:21:56,440 --> 00:21:59,880] (cue 298) We can point back to what went wrong, and we can take action,

[00:21:59,920 --> 00:22:02,860] (cue 299) and we can place the responsibility where it has to be.

[00:22:04,440 --> 00:22:09,760] (cue 300) All right. So there are several questions that this approach help us answer.

[00:22:09,800 --> 00:22:12,780] (cue 301) Right. So for example production broke.

[00:22:13,240 --> 00:22:17,960] (cue 302) And so if production broke and everything in production was built by a factory.

[00:22:18,000 --> 00:22:22,800] (cue 303) Then the question is which factory is responsible and which step?

[00:22:22,840 --> 00:22:29,600] (cue 304) Precisely because we have this receipt that chains the outputs to the evidence.

[00:22:29,640 --> 00:22:35,120] (cue 305) Then we can trace back from the source code back to the receipt and back to the

[00:22:35,160 --> 00:22:39,560] (cue 306) factory, thanks to the versioning, which starts everything because it assigns an

[00:22:39,600 --> 00:22:40,660] (cue 307) identity.

[00:22:41,920 --> 00:22:46,640] (cue 308) Another question we can answer is hey, a skill in the open source world that we are

[00:22:46,680 --> 00:22:50,000] (cue 309) using was compromised. So who's exposed?

[00:22:50,040 --> 00:22:54,880] (cue 310) Just look at the agent bill of materials or every single receipt that you stored.

[00:22:54,920 --> 00:23:00,240] (cue 311) Right. And then find which factories are using those and patch it,

[00:23:00,280 --> 00:23:05,120] (cue 312) revoke it in the rules. The gate will start forbidding any output from those

[00:23:05,160 --> 00:23:08,620] (cue 313) factories from going to production until you fix that.

[00:23:08,920 --> 00:23:14,680] (cue 314) All right. Okay. So then next question the checks pass the checks that we have

[00:23:14,720 --> 00:23:18,560] (cue 315) defined in the contracts, which are exactly our our quality bar.

[00:23:18,600 --> 00:23:22,400] (cue 316) Now they passed but did the agent change them right.

[00:23:22,440 --> 00:23:26,560] (cue 317) We have seen agents gaming the checks consistently.

[00:23:26,600 --> 00:23:29,160] (cue 318) So hey checks lives in the contracts.

[00:23:29,200 --> 00:23:35,040] (cue 319) They don't. They are not expected to be created by the same agent who work in the

[00:23:35,080 --> 00:23:38,040] (cue 320) factory. So they cannot be changed.

[00:23:38,080 --> 00:23:41,360] (cue 321) They are cryptographically tied into the receipt.

[00:23:41,400 --> 00:23:47,640] (cue 322) So if somebody tampers those, you are going to find out and they are enforced in the

[00:23:47,680 --> 00:23:50,120] (cue 323) gate with the audit command. Right?

[00:23:50,160 --> 00:23:53,460] (cue 324) So it's not possible to tamper those.

[00:23:54,000 --> 00:23:58,480] (cue 325) All right. And so let's imagine hey we want to add new checks because of course the

[00:23:58,520 --> 00:24:03,660] (cue 326) checks at the end become here our bar for automate for trust right.

[00:24:04,040 --> 00:24:07,020] (cue 327) So how can we ensure that they become mandatory.

[00:24:07,480 --> 00:24:11,120] (cue 328) Well if we create new checks these changes the factory.

[00:24:11,160 --> 00:24:14,940] (cue 329) And so we can basically release a new factory version.

[00:24:15,240 --> 00:24:18,400] (cue 330) Right. If we change the checks which are tied in the contracts,

[00:24:18,440 --> 00:24:20,680] (cue 331) then we have a new version of our factory.

[00:24:20,720 --> 00:24:25,320] (cue 332) If we create a new version of our factory, then we can go to our policy file and

[00:24:25,360 --> 00:24:32,120] (cue 333) say, now we are only going to accept diffs coming from factories

[00:24:32,160 --> 00:24:34,480] (cue 334) above version 2.1 for example.

[00:24:34,520 --> 00:24:41,340] (cue 335) Right. And so our apmx audit command, the gate will forbid any output coming from

[00:24:41,760 --> 00:24:45,620] (cue 336) older factory versions. All right.

[00:24:46,440 --> 00:24:49,920] (cue 337) And so well there is another question that we can answer.

[00:24:49,960 --> 00:24:52,880] (cue 338) You know, in the policy we can also define, hey, what are the skills that the

[00:24:52,920 --> 00:24:55,080] (cue 339) factories can use are allowed to use.

[00:24:55,120 --> 00:24:58,260] (cue 340) Again, we have that policy in the rules.

[00:24:59,300 --> 00:25:02,600] (cue 341) The dependencies are tied in the manifest in the lock file.

[00:25:02,640 --> 00:25:07,740] (cue 342) We can check with the audit command and we can get based on that.

[00:25:08,080 --> 00:25:13,340] (cue 343) Those are just some examples of what this approach can help us with.

[00:25:13,800 --> 00:25:20,240] (cue 344) So now if I go back to the factory runs, I can see on the copilot run

[00:25:20,280 --> 00:25:22,920] (cue 345) that the different contracts were run.

[00:25:22,960 --> 00:25:26,620] (cue 346) Planning a specification, build a review.

[00:25:27,080 --> 00:25:33,260] (cue 347) What allows the state to flow through the contracts is basically files,

[00:25:33,440 --> 00:25:34,920] (cue 348) right? So it's markdown files.

[00:25:34,960 --> 00:25:37,520] (cue 349) It's diffs. It's whatever files that go into disk.

[00:25:37,560 --> 00:25:41,280] (cue 350) So a contract generates files and under contract will consume those files.

[00:25:41,320 --> 00:25:48,000] (cue 351) Right. We see it completed and we can see the outputs it generated the receipt that

[00:25:48,040 --> 00:25:51,880] (cue 352) was generated in total in an in-toto file.

[00:25:51,920 --> 00:25:58,500] (cue 353) And I can run the audit command quickly to show hey

[00:25:58,600 --> 00:26:02,780] (cue 354) now eventually if we run this audit command which should run in CI/CD,

[00:26:04,400 --> 00:26:09,680] (cue 355) can we check that the integrity is there, that the provenance is the one that we

[00:26:09,720 --> 00:26:13,120] (cue 356) that we decided to trust that the checks ran, that they were valid,

[00:26:13,160 --> 00:26:16,540] (cue 357) etc.. So we can see that happening there locally.

[00:26:16,640 --> 00:26:19,920] (cue 358) Of course here we didn't sign the receipt.

[00:26:19,960 --> 00:26:26,440] (cue 359) So the check doesn't pass. Eventually what we need to do is to add

[00:26:26,480 --> 00:26:31,300] (cue 360) that add that command as a check to a pull request.

[00:26:31,360 --> 00:26:35,700] (cue 361) So I have here a pull request generated by the factory.

[00:26:36,880 --> 00:26:41,420] (cue 362) And as a check I have a single one, the audit command.

[00:26:41,640 --> 00:26:46,736] (cue 363) We should have also code scanning and stuff like that which runs the apmx audit

[00:26:46,776 --> 00:26:52,960] (cue 364) command. And the basically all the checks pass the receipt is signed.

[00:26:53,000 --> 00:26:56,920] (cue 365) It was signed by the environment which run the factory.

[00:26:56,960 --> 00:26:59,800] (cue 366) You know it can be CI/CD, it can be our Kubernetes cluster.

[00:26:59,840 --> 00:27:02,520] (cue 367) It can be wherever you decide to run your factories.

[00:27:02,560 --> 00:27:07,720] (cue 368) And that becomes a valid statement that allows us to say, okay,

[00:27:07,760 --> 00:27:09,800] (cue 369) can we auto-merge this now? Right.

[00:27:09,840 --> 00:27:13,580] (cue 370) And we have traceability and all those properties I spoke about.

[00:27:14,920 --> 00:27:21,460] (cue 371) All right. So I'm going to close by saying we we seem to be all converging

[00:27:21,520 --> 00:27:28,280] (cue 372) into something like some sort of definition of the smallest unit of what a factory

[00:27:28,320 --> 00:27:31,600] (cue 373) is. You know, Tessl was speaking yesterday about loops.

[00:27:31,640 --> 00:27:34,880] (cue 374) We have a lot of vendors speaking about automations.

[00:27:34,920 --> 00:27:36,780] (cue 375) I mean, speaking about contracts.

[00:27:37,880 --> 00:27:40,880] (cue 376) What I do think is that we need a shared format.

[00:27:40,920 --> 00:27:46,600] (cue 377) If we do want portability of of software factories, we need a shared format,

[00:27:46,640 --> 00:27:50,960] (cue 378) a shared standard, and we need not to reinvent the wheel.

[00:27:51,000 --> 00:27:55,660] (cue 379) We created a lot of this fabric of trust in the past for classic software.

[00:27:55,720 --> 00:28:00,280] (cue 380) Let's not reinvent the wheel and use the lessons learned to build the same fabric of

[00:28:00,320 --> 00:28:02,960] (cue 381) trust for agentic software factories.

[00:28:03,000 --> 00:28:05,740] (cue 382) And let's do that in the open, right?

[00:28:05,960 --> 00:28:10,160] (cue 383) So we need to define factories as code in an open format.

[00:28:10,200 --> 00:28:15,180] (cue 384) We need to version them because it all starts with versioning which is identity.

[00:28:15,240 --> 00:28:19,600] (cue 385) And we need then we can then get on evidence on actual evidence.

[00:28:19,640 --> 00:28:23,140] (cue 386) And and we get the traceability, the provenance, the portability.

[00:28:23,360 --> 00:28:26,020] (cue 387) We skip CI we skip peer reviews.

[00:28:26,360 --> 00:28:30,760] (cue 388) And I think this is how agentic software factories can scale to large regulated

[00:28:30,800 --> 00:28:36,360] (cue 389) enterprises. All right. So this is a proposition for an open format that I have

[00:28:36,400 --> 00:28:42,880] (cue 390) published. It's a draft I think this is the perfect conference to invite you to

[00:28:42,920 --> 00:28:44,940] (cue 391) participate in the conversation.

[00:28:45,040 --> 00:28:50,320] (cue 392) I hope that from there, we'll be able one day to say, hey, we're going to change the

[00:28:50,360 --> 00:28:55,260] (cue 393) payment systems in the bank and we can even auto merge.

[00:28:56,000 --> 00:28:57,480] (cue 394) All right. Thank you very much.

[00:28:57,520 --> 00:29:01,840] (cue 395) With that, I leave you with the QR code for all the content.

[00:29:01,880 --> 00:29:05,660] (cue 396) Thank you. And any questions. I think we have one minute.

[00:29:06,120 --> 00:29:08,140] (cue 397) All right. Thanks.

[00:29:10,290 --> 00:29:13,600] (cue 398) Thank you. Thank you. Talk to you about the contents of it.

[00:29:13,640 --> 00:29:15,680] (cue 399) Thank you. Daniel. That was an amazing session.

[00:29:15,720 --> 00:29:18,960] (cue 400) Thanks. You're going to be doing questions now or do you want to stage?

[00:29:19,000 --> 00:29:22,000] (cue 401) Are you have time? Then I can pick a couple.

[00:29:22,040 --> 00:29:24,120] (cue 402) Of quick questions you want to ask you now.

[00:29:24,160 --> 00:29:27,180] (cue 403) Oh, there we go. I'll come down.

[00:29:29,320 --> 00:29:31,660] (cue 404) Let me go down. I can do the exercise.

[00:29:33,240 --> 00:29:37,280] (cue 405) So as part of the checks that you're defining is part of these kind of these agents.

[00:29:37,320 --> 00:29:39,840] (cue 406) First off, I love the kind of like, OPA design of all of this.

[00:29:39,880 --> 00:29:42,100] (cue 407) I think that makes a ton of sense.

[00:29:42,160 --> 00:29:48,280] (cue 408) But in regards to the checks, are you kind of alluding to kind of them like almost

[00:29:48,320 --> 00:29:51,340] (cue 409) being a replacement more for like unit and integration tests?

[00:29:51,760 --> 00:29:57,360] (cue 410) In that or is it more a specific kind of like a volatile of the specific code that

[00:29:57,400 --> 00:30:01,050] (cue 411) you're producing as part of that individual agent execution?

[00:30:01,090 --> 00:30:06,080] (cue 412) Yeah, yeah. So the checks have to be deterministic.

[00:30:06,120 --> 00:30:10,720] (cue 413) In my view. The checks that you define in a contract ought to be deterministic

[00:30:10,760 --> 00:30:15,880] (cue 414) because that's that's basically the harness around the model that you can basically

[00:30:15,920 --> 00:30:22,880] (cue 415) trust that elevates trust. So a check could be hey what is if we have a contract for

[00:30:22,920 --> 00:30:27,360] (cue 416) writing specifications, then maybe there is a specific format in what you want to

[00:30:27,400 --> 00:30:29,960] (cue 417) make sure that the specifications are written right.

[00:30:30,000 --> 00:30:35,040] (cue 418) So that could be one check if you have a contract for performing pull review pull

[00:30:35,080 --> 00:30:37,640] (cue 419) request reviews, then maybe it's about the format.

[00:30:37,680 --> 00:30:44,400] (cue 420) Maybe if you have one for building diffs actual code, then maybe it's about

[00:30:44,440 --> 00:30:49,840] (cue 421) hey, you can attach this piece of code or you have to follow this style.

[00:30:49,880 --> 00:30:52,540] (cue 422) So linting and stuff like that, right?

[00:30:52,630 --> 00:30:53,430] (cue 423) Yeah.

[00:31:03,880 --> 00:31:06,390] (cue 424) Staging and so on. And so

[00:31:09,300 --> 00:31:12,240] (cue 425) to load on CI. Exactly. How that

[00:31:24,480 --> 00:31:29,000] (cue 426) emerged. The idea is that you skip this running the checks again in CI because you

[00:31:29,040 --> 00:31:31,040] (cue 427) have the evidence that they have run.

[00:31:31,080 --> 00:31:34,520] (cue 428) You have trusted evidence sign that they have run and they have passed.

[00:31:34,560 --> 00:31:39,520] (cue 429) So the idea is that you can you you don't need to run them again on CI centrally.

[00:31:39,560 --> 00:31:44,220] (cue 430) So shifting left testing as much as possible in a in a trusted way.

[00:31:44,970 --> 00:31:49,140] (cue 431) Yeah. You go one more time for sure.

[00:31:50,480 --> 00:31:54,440] (cue 432) Hello. Waterfall was bad idea for humans.

[00:31:54,480 --> 00:31:57,660] (cue 433) Why would waterfall be a good idea for a AI?

[00:31:58,160 --> 00:32:00,720] (cue 434) Sorry. Can can you? Because they held of echo.

[00:32:00,760 --> 00:32:06,000] (cue 435) I didn't hear you well. So waterfall was bad idea for humans?

[00:32:06,040 --> 00:32:10,340] (cue 436) Yeah. Why would waterfall be a good idea for AI?

[00:32:11,800 --> 00:32:17,120] (cue 437) So, you know, I think waterfall did very well in some situations.

[00:32:17,160 --> 00:32:18,680] (cue 438) It still does well in some situations.

[00:32:18,720 --> 00:32:21,160] (cue 439) And I agree with you that it's not a solution for everything.

[00:32:21,200 --> 00:32:24,580] (cue 440) I think your question comes from the graph that you've seen,

[00:32:24,680 --> 00:32:27,560] (cue 441) which seems to be a waterfall approach.

[00:32:27,600 --> 00:32:33,000] (cue 442) I do think we are going to see forms in which there's nothing to do with waterfall.

[00:32:33,040 --> 00:32:39,680] (cue 443) Even when you do a scrum and all this, you follow a certain approach to

[00:32:39,720 --> 00:32:44,720] (cue 444) something. We are speaking about automation here, so I agree with you that waterfall

[00:32:44,760 --> 00:32:45,720] (cue 445) is not the answer for everything.

[00:32:45,760 --> 00:32:47,620] (cue 446) And this was just one example.

[00:32:47,760 --> 00:32:52,800] (cue 447) I think we can very likely be able to model another sort of process in the form of

[00:32:52,840 --> 00:32:54,180] (cue 448) contracts as well.

skills

talk-meppiel-versioned-agent-contracts

tile.json