Transcript — Building Product Teams in the Age of AI

Speaker-label warning. This transcript has no per-speaker labels. Christopher Batey is the sole speaker for the body of the talk; the opening lines (and a brief closing hand-off) are by host Katie Roberts on the Latent Space stage. The transcript contains visible speech-to-text artifacts (e.g. "Thanks very much, Dave" appears to be a transcription error; "covet compiled" likely means "code it compiled"; "subsidy zero for funding" is garbled; "haptic AI" near the end is unclear). Preserve these verbatim — do not silently correct. When attributing audience reactions (poll shows of hands), do not invent specific attributions; the transcript only records aggregate hand counts.

No participants list was supplied beyond the speaker and host.

§ 1 — Intro & company context

Hello. Welcome to Latent Space. My name is Katie Roberts. I've been hosting on this page today. Right up next we've got for us Christopher Batey. He works for as a CTO for Core Engineering Consulting Group and his talk is building product teams in the age of AI, what we have to relearn every quarter.

Thanks very much, Dave. What I'm going to go through is an everything that we've learned, well not everything, but certainly the important things for building a product team in the age of AI. So I work for Core Engineering Consulting Group and there's a keyword in there called consulting. In historically we have been a consulting firm in the domain and platform engineering and continuous delivery. And if I see where we're at about two years ago, the majority of the engineers at our organization would be at client sites and we regularly work with like large enterprise clients across media, finance, healthcare. We do two types of things. We do advisory, so we help them, we look at their continued delivery practices, digital transformation, building developer platforms, and we give them advice on how to move forward. And then we provide the engineers to go in and tattoo that. Thing. They can either take up and do that themselves or we could go in and help them.

And for people who've worked this type of knowledge based consultancy, you know, the way that you operate as a business is because you're specialized in your domain, which in our experience, a platform engineering and continuous delivery, you can normally add quite a lot of value because the engineers at the our clients, what they're focusing on media or finance or healthcare. But anyone who's tried to scale an organization like this beyond a handful of engineers knows that it's really, really hard because as you get more and more people, it's really hard to keep your edge about being experts in that domain. So you start to build systems for it.

And two years ago, this is what our systems look like. So we'd obviously train our engineers internally. You know, the actual type of training, it would be information on wikis, it would be decks, we'd run workshops, we do boot camps with our engineers. The other thing we had was a lot of text and spreadsheet based knowledge on what good looks like in platform engineering and continuous delivery. So it'd be what features does a developer platform have, how what type of testing are you doing in your path to production? And we now have quick ways of evaluating whether what people were doing based on where they want to get to. And then the third was like reference material. So because we kept on building the same type of thing over and over again at our clients, then we'd end up with road maps, you know, predefined roadmaps with ethics and milestones and tasks for building certain types of things. And that naturally led to reference implementations. So the key features that we kept building and building again for different clients, we'd end up with some kind of preference implementation we could bring in.

§ 2 — Where they are now (with AI)

That is not where we're up to date. And the premise of this talk is we probably wouldn't be where we're at today without AI. So if I take the training that used to be wiki's decks and workshops, that right now is an internal interactive training platform. So all the engineers are drawing our company, they go learn about continuous delivery and platform engineering. They learn about concepts like versioning and promotion or different types of testing so you can make money for software more quickly. And then we have software which provisions that are realistic environment to try that skill out. So your training is normally made up of like showing something, you know, telling something, but now we have an interactive training platform that can do the thing. So like an example would be it for a bit automatically provisions them, a GitHub repository, an environment like a namespace, Kubernetes plus cloud account. And then the software running in the background, which is checking that they've done the thing, you know, like deployed in a mutable version artifact.

The processes, which used to be lots of wiki pages and spreadsheets and all engineers love wiki pages and spreadsheets is now like an interactive tool with all of our engineers use to track the maturity of various things that we are specialists in all of our clients. So it could be security, it could be security, it could be application architecture for continuous delivery. It could be the types of testing or promotion you did.

And the reference implementation, so the roadmaps and developer platform features, well, we still use those with enterprise clients, but we use them to build an off the shelf developer platform for like smaller companies. So rather than just working with huge companies, we now work with small companies where we can install this into the cloud provider and they can get some of the benefits that the big companies would normally spend billions and billions of pound building internally.

And just to add on some more fun, that internal tool has become a SaaS. So now our actual enterprise clients can log in themselves. And reasonably complex to do that. And then the product which we built, which is for small to medium enterprises, well, we're actually its best customer because like when we built it, I wasn't thinking we're going to use this to build all of our products, but now we do. So we use that as a customer.

And just to like give you a flavor of what this looks like, well, one of the managed services which we host, this is imagine, this is what I mean by insights. So being able to quickly see the maturity of various stages of a pasta production across lots of different components. And then, you know, the developer platform, this is the user interface for it where I can see which version or which of my components is in different environments as part of the path to production. That's not that interesting. Okay, that's just what we build.

§ 3 — The stakes: sensitive systems

I think the important thing is we're building two types of systems, right? There's systems which manage very sensitive data. If any of our clients were to need the data which we have in the first system that I described would be leaked or exposed in some way, that will form you at the end of our relationship with that client. And then the other product manages people's production environments. So the impact of it going wrong is pretty severe. So that's kind of the impact I want to talk about. I'm talking about building a product team which is going to manage a system where it really matters if it goes wrong rather than like working on lots of other small projects which might be like websites with basic functionality.

§ 4 — The fear: software no human has seen reaches production

And that is in the context of everything we're hearing today in an industry and especially on social media that we're rapidly approaching a world where software, no human has ever seen reaches production. And I don't think this is hypothetical anymore and I'm definitely not talking about our side projects or like one person businesses where they're quickly ideating and putting something in production. I think that's totally fine. I'm talking about the systems which have your credit card details or they manage or in your personal healthcare data.

And this actually starts to scare me, right? Because code. Can do anything with the data it has access to. And if we're not looking at that code, they could do anything. So certainly if my bank started doing this or my healthcare company started doing this, I'd start to be pretty scared.

And I think some of the optimism is real. All right. Everyone who's kind of worked in this area for the last 24 months, especially the last nine months, the last 12 months, AI has become absolutely fantastic operating. When we started building these products, it was 50/50 whether the covet compiled with these tools. Right now it's going to compile. It's probably going to work and I'm more just spending time refining how it works. So it is pretty amazing.

That said, I don't think this is good at other parts of the software delivery life cycle. So in question for you to think about is writing poem the hardest part of software engineering because I definitely think that's the best thing right now that AI is good at and is software engineering really the hardest part of a successful digital product and building and building products. By which I mean like adopted features in production that potentially generate revenue or whatever impact.

§ 5 — Audience polls

And I wanted to get, I'm sure this isn't going to be the only time you're going to be asked this today and I've seen this been asked quite a few conferences up until now, but I wanted to get an idea of who's in the audience. So for the code or digital product that you're working on that is what you're paying for. I know we've all got like at least 20 side projects recording on at the weekends, not those. What percentage of your phone is currently written by ai. And it's good to exercise. I'm going to get you to put your hand up.

Who here has less than 10% of the quote written by AI for their real system. So we've got one, two. Any like full manual code that's an input zero as an option. No, no, but under 10%. What about under 50% So let's say maybe 20%. What about 90? On the 90 100% There's quite a few with 100% We're there.

I would say for us we're at 90 to 100%. It's very, very rare in my company that any of the engineers actually write some code. I would say though that this is a vanity metric which people like to talk about because out of all that code and if I look at my main branch of any of our products, it's all written by AI now. But how much of it was written without any human intervention? With software engineers like using critical thinking. And I think the answer is probably like zero for subsidy zero for funding.

§ 6 — AI as amplifier

I would say that my experience over the last year or so if AI is an amplifier. So for the things that we did really well as a company, it allows us to do it much quicker. Right. So the things that we did really badly, which we're trying to improve on are many of the like the topics of this talk, then all it's done is amplified the things that we didn't do so well. For me it's really enabled like delivery focused engineers. So people are really focusing on getting working software adopted in production is really like within a game changer for us. And my company certainly wouldn't be in the position it is right now if AI didn't exist because you know we couldn't have afforded probably to hire a 10 person like product team to two years ago.

What about this question which is slightly more controversial and we're going to see if we get to the same kind of answers. So what percentage of goes to production without a human ever seeing it? So the person prompting the agent, the automatic agent workflow or maybe someone doing a review after that. So there's anyone have no code going to production. Actually that's more than 50ish percent. So I'm pretty happy with that one actually. Does anyone get all the way to 100% where not a single human sees the code which goes into production? Is anyone in the other extreme? I guess we know there's a couple, there's a couple. Well it's good. Gives me hope.

We're here. I would love to say actually zero percent even though like we're massively adopting AI and we must at my company. I just couldn't guarantee it so I won't put the less intense. And I think the thing which like worries me more and more important is what percentage of your code already what percentage of your system goes to production without human understanding. It.

And to think about that I wanted to like dig into and start to think about what we actually do to build a digital product. So we've produced code for a long time digital assets and we end up with a product. And the producer is up until recently been a human, you know flesh and blood. You know product owners QA's developers. It was a team. And the thing we produce is a digital asset right? It's code. It's an artifact. It's verification. It could be the testing. It could be monitoring alerting. It could be the documentation. So this is where we're at a couple of years ago.

Now the production of our digital product is very much becoming a black box to all of us. Right. So you could be using some identification for code. Some kind of autonomous agent. But we I'm not too worried about this right because I'm still producing my digital product which I own at the end of it and I understand it. But of course it's a slippery slope which we're all being pushed into doing it. Go faster. Go faster. Go faster. So at some point it becomes this. When we don't understand really how it's being produced and we don't actually understand the thing that's being produced.

But don't worry because obviously the answer to this is we look at the verification. Right. So as long as you understand the black box tests of the thing now no longer at the sound it's okay. Right if I understand say my acceptance test, my functional test is doing the right thing. Then everything's okay. But when people do this what I find is it quickly becomes this and quickly becomes this then quickly becomes this.

§ 7 — Don't delegate systems thinking to agents

So my first tip and the thing which we've tried to work on recently inside my organization is to not delegate our systems thinking to agents. So I'm not growing this part to say everyone understands every single line of code, but let's not delegate systems thinking to agents.

So I want to give you a concrete example of a change in our delivery process that we made recently. So here's a very simplistic architecture diagram of one of our managers services. We've got private backends like training pulse insights teams. We've got public front end and some kind of authentication service. What it does doesn't really matter. But this is something I clients just log into. They have no idea how it's implemented and if they have a new how it was implemented it's probably because we broke it somehow.

On the other side is something we deploy actually our clients. And this is an intentional like system design because it has access to their data. It does lots of processing where the clients would not want that data to leave their premise, their network. So it has its own front end.

Now lots and lots of features keep on getting requested that require this connection. Right? Some of the functionality we've got in the managed service and then combining it somehow with clients like agent which is running and collecting and processing data. Now I could just describe the feature I want. Right, the feature I want which would combine this and I'm pretty confident that our identic workflow would build it. And then maybe a pull request is going to land which is maybe 7,000 changes. And then what would I be reviewing it for?

Well I first want to review it for the systems thinking. So here's some questions which I just threw together about what I want to know about this change. You know how does the client side connect to the managed? What does it connect to? How does it authenticate and authorize because this is a client side component talking to a multi-tenanted managed service. What parts of the managed service are exposed to the internet? We're very careful about not exposing things. You know do we ever share an API which is used by human usable front end versus like an authentic system? What data compassion applying to the managed? What data can possibly manages the client there are different deployment mechanisms. Right. We do continuous delivery virtually every commit for the managed service for the client side. That means that we need to do something on our client and maybe that might get updated once a week. It's deployed to their systems. So that's the type of thinking I want. Definitely want to keep into the engineers that I work with.

§ 8 — ADR-first workflow with AI

So one thing we do now is when we're planning whatever work we're going to do in the next week is if we think there's a system level decision that always comes as a completely separate pull request in something called an architectural decision record. You've not come across those google them. It's about documenting your technical decisions along with your code so that you can someone can quickly learn why we made the decision so we don't keep on discussing them again.

And we always use ADRs or at least for a very long time. And when we first adopted agentic development over the agents loved writing ADR they're very good at seeing that we had ADR so they wrote more ADRs and our ADR is much longer more verbose and not a single human understood them which completely defeated the point of having them.

So what we did is we said from now on we're going to start with the ADR. I'm me as a human, I'm going to spend a lot of time reviewing it. I want diagrams. I want them to visual so I can understand them. And then humans reviewing them can really put a lot of effort into reviewing those architectural decision records. And once you've got those. Agents are really really good at reviewing implementations against a well structured architectural decision report. Not only when you eventually land in the patient, but all of the follow-up requests then because one thing I don't think humans are good at is keeping all of those ADRs in your head and then subsequent pull requests which subtly perhaps change your system architecture. You might not realize that something's happened like a new API is exposed. But what we actually do is have the ADRs constantly reviewed against all the world requests that are coming.

So that's kind of my first tip about not delegating systems thinking to agents.

§ 9 — The producer "black box": harness, host, model

The next thing I wanted to dig a bit deeper into is have a real think about the dependency you want to put on the producer of your digital product. So we go back to this diagram. And we've accepted that maybe we don't understand how identic workflows actually produce code. If we were to break it down a bit more, so I'm breaking down the producer box into three things. We've heard the first one. I pulled the interface but in most places we'll call it harness like in the keynote. So there's an interface for how you do that. That could be core code. It could be open phone. It could be codecs. It could be some identity automatic workflow based off your, you know, your GitHub issues. Then you've got the thing which is hosting your model. Right. So that could be directly it could be the on-topic API, but you could be hosting your own, you could be using a third party which I feel needs to reach between them. And then you've got your model. So really your back box is made of three things that you don't understand. It's called permanent API which perhaps at 3:30 on an afternoon starts working. Anyone experience that? Does anyone feel the most productive once the Americans wake up? And then you've got the model which honestly you don't understand. I'm not saying you obviously understand it. If anyone wants to explain how all this works, I'm very happy to hear it.

So we can choose not to be quite so comfortable to that workflow. We can use a harness which is open source. What's the benefit of that? You know when Anthropic goes down I just switch to open API with a single command and I carry on. I obviously get different results. Every model gives you different results but it's good to have your way that you're building your digital product be able to flip between buyers. You can go as far as the hosting for models themselves. That limits the models you can do. You see. And then if you want you could use a model with open weights or at least an open software that the weight is trained.

I don't think any of that is that important apart from being able to switch between providers very quickly. You've got a production outage and suddenly at the same time that's in your favorite AI provider does and the only way your engineers can debug productions with that model good luck.

I think it's more important that you end up understanding the product that's developed. If they can't take that away from you I don't think. Right. It produced that you're an intellectual property. It's a whole different ball game though if you build out that interactions into your product. Then you've got another black box inside your black box. So. Vendor lock in is fine as part of your development process. Maybe we've all done it. Cloud providers all sorts of things but make it a conscious choice in your organization and understand the consequences of that producer going down at key points for your business.

§ 10 — The whole flow: idea → product mgmt → ownership → build → adoption

The next thing I wanted to dig on is the changes we've made to the flow of how software development happens all the way from idea through to working software interaction and actually one step further. So I'm talking from now on about like an existing product with real users with real data so we're not talking about like an existing like greenfield like prompt off you go. And it's not users.

So we have product management, we have ideas anyone can have ideas in the company about how we've involved the software. We have product ownership so that's coming up with a concrete plan for turning those ideas into taking what we've got and modifying it and ending up with it. Hopefully okay on the product. We've got people who build it, actually go off and execute it with all the new tools. And often the step which is forgotten about which is driving adoption and evaluating whether the idea was actually good idea. Right. Because every time we change our product, our digital product, it's an experiment and there's no point doing experiments if you don't actually look at the result and then learn from it.

So all of this should feed back into that process and that is the amazing thing about identic development because some feedback can be actioned right away and if you have a good pattern for the coin software as your users are starting to adopt it, if you're sat with them perhaps if you're a client you can actually just change the software in real time and how they're deployed through your after production.

And one thing that we noticed is the different stages of that path to production had different speed ups from AI. So this is just our experience. Yours might be different. The buildings where the order of magnitude has gone right. If you give a senior software engineer like the time to just build features, they can build as many features as you want right now. They might not be the right features.

If we go to the product management product ownership we heavily use AI there too to do product market research to analyze user feedback to build roadmaps to refine issues. But there's a lot of decisions to be made. There. Right. There's a huge number of decisions. It's much more important to decide what we're building in more order and that is not something I delegate. So that's why we haven't found we got the same speed up as we did in building.

And this one for a while is actually for harder, right? Because we're providing too many features to our clients sometimes in a great problem to have isn't it, which can make adoption harder because beat dropping new features before the previous ones. So I think look for friction in your process is caused by different levels of acceleration. And you work out what your flow software development is.

§ 11 — "You build it, you run it, you drive adoption"

And one kind of tip which we've recently executed on is to have software engineers be directly responsible for adoption of the features they build. Do not let them just go build the next feature because you end up with lots of unadoptive features which is essentially useless. And if you don't do something like this or you don't speak a manager in the same way that you build them, you have to expect software engineers to build features that you do not need because it's fun. Rights is the development is fun and the friction for an idea turning into working software has gone down.

So we often end up. The way I think about it right now and how the software engineers work in my team is that we've merged together the friction of getting adoption with the building. Right. So engineers aren't allowed to go build the next feature until they've done everything they possibly could to get that feature adopted.

And I come from a software development background that became a platform engineer and I really believed in the cultural side of DevOps. And you build that you product with something which I would set for many, many years and I really believe in. But for now it's like you build it, you run it and you have to drive the adoption of that feature. Otherwise you just send them too many features, unused features in production.

§ 12 — Engineer day-to-day: plan / execute / refine

Now I'm going to dive a little bit deeper into how the role of our software engineer changed over the last year. And I'm going to break down what they do in roughly three steps. So we're now just in the build box of the previous diagram.

So technical planning is the first part of like actually building a feature architectural decisions, testing strategy, you know how to integrate with third party dependence. Is that really, you know, you're using a lot of your brain. The brain is a software engineer. It's absolutely the thing we put into ADRs that I talked about at the start.

Then we come to execute on set plan. And obviously that's the bit that's got extremely good with AI, so what used to be light and light.

And then you refine. And we're familiar, the refinement is where the magic is up with AI because we can quickly iterate on our ideas based on the user feedback. If people don't use things like superpowers, I would highly recommend it. That's the one that we use. The bunsey speculated driven frameworks that people use. But this like kind of matches the brainstorming would be the technical planning. But don't delegate to it. I'm going to ask you lots of questions. It's going to make you feel very informed, but it's up to you to come up with pick the right technical design. And then obviously the writing and executing plans if you haven't broken yourself by developing a flood.

So one of the key changes in how an engineer spends the time is previously I just spent a short amount of time doing that finding quite a long time relaxing with my cup of coffee coding and kind of like for a pre-tech and then doing that pallet into deploy. And most people feel like a superhero initially, right? Superpowers is a very up snake because I think we're going so fast with any other.

At the same time anyone find AI being frustratingly slow. Right, I've come up with the design. I told you how to implement it. Why does it take 30 minutes for you to go and like do that 4,000 line change? Come on AI. So what do we actually do? All right everyone's probably gone and done this. That's my flow. I've compressed the coding. So what do we do once we've done the first technical planning? Any guesses? Yeah, we start another one. And then we start another one, right? We all end a very, very tired.

§ 13 — Parallelism — but one complex task at a time

So parallelism is definitely necessary. In our engineering hardware, we actually like we almost mandate to everyone in my company is assigned multiple tasks at the same time. But we have to be careful about it because what we found happened is if let's say people had three complex tasks. Like flu features which require your brain to think about system architecture. The first one would be fantastic. The second one would be that's a bit. By the time you got to the third one would be a load of rubbish. So even though we hadn't delegated our systems thinking to AI, we've already used the power brain so it's going to come out a bit rubbish.

So you know one of our kind of guiding principles is to work in parallel. Absolutely, but only work on one complex. Task at a time. And we started assigning to label issues and things and say yeah you can pick up these to like go while the agent are doing something else.

§ 14 — Team shape: two-to-four sub-streams

Everything I've said so far is about the individual, right? But I'd say the hardest part of this our new way of building software is how to get a team to work together to solve problems with AI. And I do think you need a team as in what I want a team of people that can operate and evolve and maintain like my digital products. I don't want one hero. I don't want some AI system that's not accountable. I want them to use AI as much as possible to build it. But you know people of the counter should be accountable have like the real agency.

So I'm sure a lot of people have heard of like Arlington's two pizza tea. You know keep team sizes down to like six to eight. Absolutely ludicrous idea as far as I was concerned because obviously you've got eight people and you need eight pizzas. I think in married computers are bigger than us. Anyway let's assume that you can't feed two a teeth and two pizzas right if you don't think about it what's going to happen is this right everyone's going to be addicted to the new game which is just a tougher development. They've got a problem. And what's going to happen? Lots of pull requests right 48 pull requests and then someone is going to tell you that the most common complaint in AI software is not as to start by development is what is it? Reviews of the bottleneck.

This is crazy. I think this is crazy right because we're complaining that we can build so much valuable software that we don't have the time to look at it. And we spent a couple of decades inventing practices to build a shared understanding of a system and maintain it. Right. We're talking about XP pairing mobbing boom runs. We did all this stuff with the goal of not having knowledge side of it.

I already think this helps right because if you put the friction of adoption directly to the engineers they don't build quite as much stuff in parallel which is a good thing because what you want is the outcome use of the software not just features in production. And like this is like I've gone through like most of the engineers I've worked with and yet you kind of get yourself worse from the features you prompt. This is not a new problem with AI with all the struggle that people want to tip it up the code get the commitment to get home rather than review other people stuff. But it's massively accelerated by AI as a big as a bigger problem.

Anyway, so what have we done about it? So rather than having our normal team sizes in six to eight people we've made them much more smaller. So they're now two almost four and we give them a mission and the remission is based on adopted software by a particular customers. And you're not allowed to start the next feature unless you've done everything in your power. So if we're on a stand up and you're telling me how great the next feature you're building is we haven't spoken to that customer about getting them to use the previous feature. That's not something we do anymore. Which then means that that small substream needs say no do the review and fix the bugs gathering the user feedback.

The most important part of this though is if we break that team of six to eight people into substreams of two to four is you get three pizzas because honestly you can't have a two tin subs you share pizzas.

The problem this produces which is the same problem that we had when we broke up bigger teams in like we broke up bigger teams into teams of six to eight is they could go off in different directions. They've got a mission, they've got focus, that's fantastic. So you really need to think about how you're going to do product ownership across these different streams. Let them work in isolation and let them just review their own stuff. But make sure the key architectural decisions are still reviewed by the right people and make sure there's at least one or two people who are working across the stream and understand how everything is going to work together.

§ 15 — Does AI need good engineering practices?

So a question I keep asking myself through all of this transformation is has AI teams good engineering practices? And you know this is a graph that completely made up with no real data. But we start off with high velocity and then we get slower. And then we fix some stuff. And then we get slower and then we fix some stuff then we get slower and then maybe we end up somewhere you know below what we started at right so we start off conquering the world and then eventually we end up with some steady pace for a digital product.

So what causes this? Well there's two answers. There's technical depth that's the obvious one and the other one is success. Obviously success is not necessarily a bad thing. When I say technical debt it's things like longer testing cycles, you know more dependencies harder to make home changes and obviously agents will have the same problems that humans have for modifying a code base in a safe way. Success is going to be things like more customers multipliedments larger data sets.

So there's certain engineering practices which I think are even more important. I've got more that I can cover in this talk but I'm just going to cover a couple probably one that I've got time for. One minute they go on the first is system architecture really matters. I already talked about this but why do you break things up into loosely coupled services maybe a very important word losing a couple otherwise you've just got to distribute the best. Right it's for fast feedback cycles. If we're going to build more features we're going to end up with larger systems. I'm talking about complex systems rather than just like say triple web apps. It is imperative that we keep the feedback of each of those components very short. It's 15 minutes for me is absolute maximum for running a full suite of tests which gives me a high level of confidence that thing works.

The other thing is if we're going to modify software more quickly blast radius really matters. What is the worst thing that can go wrong in one instance of a pipeline after reduction happens and the only really way to do that is to design those systems and so that if one of these components goes down most of your product functionality still works.

And I was going to go through an example of the types of tests we use but I'm not going to have time but feel taught in me after it if you want to.

§ 16 — Closing — find your principles

So I'm going to end on time with a few final thoughts which is to find your principles for using AI and by that I mean stuff at the start. Decide what you want humans to understand versus agents. Decide like what dependency of vendor lock you're happy with. Identify your real bottleneck all the way from idea to adoption. Do not get sucked in by vanity metrics that measure one little bit in the middle like token usage or commit or PRs merge. It has to be adoption. Features in production is what I used to use before AI. That's not good enough anymore. You have to just actually have to be used and you have to expose those to your engineers. What's that for what we're trying to find your definition of goods in a very concrete way and then how AI be held accountable to it.

I'm turning this talk into a few articles for our website if you want us to send you them just as a QR code there because we love QR codes these days and they're better will come back. And thank you very much for taking the time to this. I'm sorry there's no questions because I'm sure there are plenty for this. I'll stay here and I'll go outside if anyone's got any questions I'm very happy to answer.

Magic right up next you've got a break. Everyone picked up swag it's just outside the door please go and visit us. Most of us need Haptic AI residents of any shape.