Transcript — From Pipelines to Prompts: Surviving the Shift to AI

⚠️ Speaker labels are absent in this transcript. The source is a single continuous block from speech-to-text with noticeable garbling. The three panelists are:

• Stephane Jourdan — Co-founder & CTO of anyshift.io
• Simon — Head of Enterprise Architecture, Saxo Bank (transcribed as "taxi bank")
• Samantha — affiliation not clearly captured
• Moderator — unnamed

When attributing quotes, prefer hedged phrasing ("one panelist said...") unless the sentence clearly names a speaker. Known garbled tokens: "taxi bank" = Saxo Bank, "snake" = Snyk, "Could you be resnick" = garbled introduction, "Sam" appears as a name, "cog" likely = "code", "yagis" likely = a name or term that was misheard.

---

Section 1 — Intros & framing

If anybody is here, but the call start session, the humans architect the system and the AI writes the code. That's been moved to the other context window. So I won't mind if you sneak out now. But I wouldn't advise it because we have an excellent panel. We are going to be discussing from pipelines to prompts survival shift to AI. We have some fantastic speak panelists who have agreed to join us very late over the years as they come up, give them a nice round of applause because they didn't know they were doing this to the lender. So could I please ask. Stephane Jourdan? Could you be resnick? And Simon rona? We've come up to the stage. So hello. I'm going to get you all to introduce yourselves and then we'll just give you a couple of minutes on the online work. A bit of your background as well. And then we can sort of kick off with that topic itself.

So Samantha, do I kick off? Sure. Hi, Simon. I work at taxi bank in Denmark. I'm head of enterprise architecture and ways of working. I'm also heading the airways into excellence. And effectively spearheading with the roads which I take development across about 700 developers.

And for me I was a co-author seem safe in. Which is a consultancy helping enterprises to move to AI. Mostly around software development, infrastructure and actually building large. Platforms. So this is my second company. The first one was container solutions. Which was kind of pretty big eventually. And yes, we're doing a lot of large platforms. It's basically context for the agent or agent introduction. And in the last 15 years, I built different startups for manager function systems. And the last one was acquired by snake. So I was in the club.

Section 2 — Is this shift bigger than past ones?

Okay, so I'm going to be looking at my fund because like I said, this is quite a late notice panel. So people in this room have lived in room more than one industry, okay? They built a plan and champion devops that altered security onto delivery pipeline with dev setups and now they're facing the next big move. They are native relevant. Things. So. All have been any major size big trick in what they're doing now. Session shifts.

I think. So we started doing cloud native in 2014 and it wasn't cold calling. It was in the beginning just containers and releases came later. So at that point. It was to us quite clear that it was taken to the transformational rule change in our organization. In what we see now with the is similar. We're not just changing the development process, but it's changing quickly everything in the organization. And it's making them a hundred times more dramatic than 12 because cloud was sort of limited to dev teams or engineering teams. What we see now is that the moment developers start using AI very quickly they run out of tasks in the backlog. And then the product owners need to pick out to do something else. And then all the other departments have to do something else. Everyone has to adopt a and I think this is something that. Is for starting my career when the piece of technology so quickly changing so much with that company.

Starting to bring things into the pipeline. Is that not a fact that is absolutely. I think although devops seems to be a bit behind the general development in others. I can guess why just the impact is more dramatic and more immediate if things are go down. But if you change the configuration, just AI and it fails. The impact is immediate. Right. So I think in our experience, devops is a bit more, it develops things a bit more afraid. They're still getting into that. But it's also not clear how AI is going to affect the maintainability of the products. Or it's probably clear that it's not going to be good.

Section 3 — AI in production / observability

We were talking earlier about you tend to look after things in production. Do you want to pick that up?

Yes. What we observe now with agents creating PR screen hours a day. Now we have like CEOs employees that function. We have faces everywhere. I'm talking about microfunder. And now everybody is deploying fixes, changes, new features all the time. Usually it was already likely just to know what the impact of not to mention just things that drifted manually or was imprisoned. So now you have agent, buy everyone from everyone. Doing things in virtual, how do you know the impact of this tomorrow's a day? It. Comes out and you really proactive agents and all the context. What changed today? What was the relationship between this service and this other service? How do you know that this specific service was actually connecting to the redis, but the variable was not the name of the IP, that sort of stuff. So it's not only about like look code analysis, it's really about like. The context. Old the knowledge you can have control, you have multiple accounts, that sort of stuff and agents today, you mentioned that they want us a bit. Back. In the space because the lobbyists have a lot of tools, a lot of. They have to work with the code, etc. But for products, we really need to. Keep the lights up. So that data that complies super important. These days, including agents fix this 24 hours a day because even like from a security perspective. All the things that are being patched as well. We need to ensure that the glass rate is all things are still working after this. So this is everything is so quickly is based that data collecting. Furiously.

So I work in a regulated industry, we can't quite move as fast, but we're still moving pretty fast. Like I said, I've been spearheading this and sort of coaching developers through it. And I think one of the things for me that surprised the developers or, you know, we do the ability running. So it is really developer operators. The thing that has surprised our developers and noticed how useful the agents are for diagnosing production. Disorders. Genuinely throw them at elastic logs plus service now incident reports plus the code. And the combination, like you said, of this knowledge that the agents can have, they will solve an issue in 30 seconds that could have taken developers hours. Or maybe even days to fully understand. This stuff is so the people who are just using it in the development context really shouldn't if they have that data available to might you say if they've got a good observability stack in place. Should start to expand their use. And I think we are living through a seismic shift. We don't know where we're going to end up. We don't know what the earthquake will do to us because the treasures are still going and they're going to be going for a while. And I think these things are fairly clear that production use of the agencies just is important.

Section 4 — Guardrails and "musts"

To Scott. We know we should have. I know that everybody would like to have not everybody has. There are other guardrails we need to think about as well. So what are the sort of. Like musts? What are the ones that we should all go if we haven't got that this is going to cause us problem.

S? I know quite depressing conversation with a fairly new token last week. We use open code on top of vertex and he was saying, oh look, I don't need to do linting because the code that comes out to the LLM is perfect. No, I then pointed him at bonita's article. Harness engineering and her latest wine. Let him, for me that's the absolute basics. The agent write for quality. Rated writing code, the loop you can get around the productivity, but ultimately the kind of qualities. You need that combination of determinism against what Gita has written that. At length. The combination of determinism for co-quality and the feedback loop. To be able to do that. So for me number one really is linting. You know you can add more and more custom lenses. You can use tools like code house to go much deeper into the quality of the care of the agent as well, but basically get your linting turned up to. The.

One thing that is pretty clear is that you cannot continue this humans for every week. Thanks for that. Yes, none about this. It's practical like in the beginning of course that's a way to ensure quality to a certain extent that creates a kind of problems which you have. You have to have senior engineers who understood the code. They have to maintain their capabilities to understand code, which is actually hard if you don't write the code. They become bugs on that, right? So if code development goes so much faster, humans cannot keep up with reviewing the code. That's why our engineering and all kind of forms of specialized agents and stuff that they are doing this, it's absolutely essential. Otherwise the productivity of single developer may go up, but of the team remains relatively stable. What I observed in such teams is also that now new services are becoming like unknown to the rest of the team because we deployed by someone who created this by an agent because like clothes decided that this was a good idea to create this new microservice and deploy it and nobody knows really how it works, like white connect or you mentioned like the logs and Telastic, why do I stick in this case was because clothes decided that so you need to add knowledge somewhere because maybe another service was used previously for this other system. So you can have like gold brains to ensure that those sort of things don't happen, but it's not vibes or items are really like a lot of heterogeneous environments. Being more and more deployed and that way you need like even more information to debug it because no one knows really these days but are actually employ.

Ed. And the information that you are giving is often. Like it has to be presented in the way that humans can understand and make the trade of the surgeons. And so often the case as it's often there's mountain of intervention and you need to go one by one. Others again not practical for a human to do the same like that. So there has to be entirely new tools to go that will show people the trade of threat. That's where people need to be involved more. Not in reviewing every wine of code. That's probably not important. But the trade of decisions cost versus speed. Architectural decisions, which kind of observability tool to connect. Those are absolutely has to be done by humans.

Section 5 — Explainability and the next abstraction

Okay. I'm interested in the sort of whilst I do get. The, we will have to get to a point where humans can't forget the code. I do remember what guy was talking about this morning, that if you lose that context of what the code is doing, that sort of problematic as well. So are we talking here that we're going to have to have some way of. Reliably skimming what we've got so that we can then I don't want that we can already get documentation. We can already get a system diagram. We can already get an awful lot by pointing an element upper covid to tell me what it's doing. Is that what we need to invest in on some things that are allowing us to get even better views on that code that's going in.

Generally explainability of the code of the systems so humans can understand make decisions. I'm not saying it's this point is foreign going forward. So we really need next level tooling. If we assume that AI is sort of next level of abstraction, then we need to move away from reviewing code and move to that next level of abstraction to new language that describes the system. I don't think that exists. So and that's why we are forced into going and reviewing every one of you and relying of operational logs and stuff. This is not practical. Like this is like going into assembly and reviewing every line of post.

I agree. Again, I just have like a mess. We're in the middle of the change here. Simon Ward leans you together quite a bit about this, about how the explainability of the code being able to understand the cog rather write it is the new skill, but I don't again talk about. Judah's worked a lot. Some building development environment stool kits is pretty impressive for some single code bases. But when you get to enter my scale there is nothing. Else. And you can't just rely on the LLM because they have a local hallucinative or miss stuff. You need something deterministic. To tell you what the code is doing.

And it's going to be very interesting. That's something to close the ISO as well is teams taking on much more production systems or applications to maintain that they don't know about. It was inherited by gene that left or an acquisition or something like this. Now as you can pick on your kids, it's not possible with the right tools to just manage these products because if you have a project, if you have like you can. Do this in like seconds now, which was just not suitable at all. That change a lot of teams are taking. Over. A lot more than.

Prisoner. S. Yeah, that's something that I see as well. You still have to follow it. You have to diagnose quite strictly because you're going to release because it will lose. If you just give it free rights, you can obviously do it. It will tell you what you want to hear. That's why you need the property of the company. This information is crucial. To Hender. Burn.

Section 6 — Hallucination and acceptable error rates

There is this element of like. We tend to compare systems to perfection. Like it hallucinates a large problem because we expect it to be perfect. I'm like, okay, but humans are perfect. They're not perfect. I mean all hallucinate in a way. We all do stupid things. So we need to compare it. It's like self-driving car, right? It's not about zero accidents. It's about less excellence than what we have with students. So. There always will be programs, but that's okay as long as we can manage that.

Section 7 — Recent AI outages — what to do

So let's actually lean into some of those problems. We've all read a few of the really good terrible things. The railway, GCB and AWS also DNS. So there's been some really. Foliage. Is there anything that we need to be doing to make sure we're not falling into the same sort of traps? And is there anything where we need to be starting to think about as part of our pipelines?

I'm being very blind in the Sam. I think it is, and again this is where what we should have been doing with humans. Literally anything goes wrong and I think this is your hashimoto harness engineer. Ing. Anything that goes wrong, feed it back into the harness and then never let that class of error happen again. Do something deterministic in your heart. If you can. Yeah. To make sure that class of owner never happens again. Feedback, feedback, feedback. Which is the same. Exactly. There's high functioning. Exactly. There's a high functioning teacher. We've been doing that classification. Should we do that type of thing anyway?

Yes. AI just picks up whatever you already have. If you have a good system that doesn't fail. Then you go faster without failing much. If you have terrible system. It's terrible faster. That's what the door is.

Yeah, and similarly what we observe is self learning agents for production. Like it fits an issue. It's just centriole data about whatever. An information. And we have like reflectors agents like learn from both the problem and the solution and it gets added to the memory as well. So let's self reflecting agent with a very specific memory and it learns what happened over time as well. That's a new class of agents for prod. That's very exciting because it really learns something very specific to the company and it can be really tailored. That's what we observe as well more to pricey companies that have huge no human can really know what's really happening on such large.

Conceptually speaking it's similar to a good healthy team doing continuous integration delivery. There are things that are running bills and whatever you have red building they do nothing about it. Are you actively fix the thing like that and now agents are doing it. But conceptually it's the same thing. Like do you have a discipline to continuously improve? If yes then you can build agents that you're doing it. But I think many companies don't have the discipline and they are just doing it as sort of ceremonies. Then everything becomes much worse. They're going to fail faster.

Yes. And that's the thing. The slow speed of human was sort of high dense problems. But the most advanced teams they explicitly lower the water, expose the underwater stones and then fix them.

Section 8 — On-call and pager duty

The other side of this kind I suppose is that if we do get everybody to a point where they are lowering the world, they're looking at the looking all those five things. Do you get it to a point where we can end on core rotors and pages you do.

Well I'm building certain products. So I'm biased. But yeah obviously you can plug certain agents to your pay-to-body alerts and at least you can start investigating you get paged by the time you get out of bed at least you can have like an agent like giving you a market flat file with the right information, the right logs of information. But the private, the internal information from the company. So yes, that's what we do. It starts to exist. It's working and I hope everyone gets something like this not to be trying to walk it up at 3am. So you at least end up with something like forefront support. So effectively the age it becomes fertile and worst case there's an historic hatch to go to the human.

Excellent. We start to observe this as well for triage. So like first the first responders were issue or customer issue gets routed the wrong team for example like agents with the right context to hit on the company. Here as well. It helps here as. Well.

I think it sounds like the dream. Yeah well it's obviously a self healing system that would be the total draper. I think the reality of it is I don't see like what's going away anytime soon. They just walk into something slightly different.

Section 9 — Two-year forecast

Okay I'm going to ask us one more question and then I might see if there's anything from the audience if that's okay. So imagine your site here in two years time. And same time or same group. What is it that what's the piece of information that you've shared today that you think you're all fucking go that was complete?

I'll probably be more opportunistic than you on the early output quality things are going so fast I'm pretty sure materials from top will. Have. Amazing photos. And yeah probably two years on regrets to say something that you've done at the moment. Because yeah maybe agents won't be a thing to yourself. What these agents if we have different systems that deploy and deploy.

I'll probably answer a different question. I think there's this. Naive way of looking at it the first like the easiest way to look into your eyes to say whatever we're doing now can be automated and done in construction of the time and 10% of the time in 1% of the time. So the main way is to say we will have just 10% of the developers and the rest can go home and forget about it. That sounds pretty stupid again. I think there's general spiral dumpsters. Clearly when something becomes cheaper then we do a lot more of that. I think the direction is that the software development is something that's going to explode. Everyone is going to build software. I don't think they have the tooling for that yet like what we call wire coating. It has to be sort of built on top of bigger platforms that they provide large scale functionality. But everyone will build their own front ends, their own sort of last mile of consumption. So there will be sort of explosion of small software. On the last mile and there will be massively bigger systems that we could never build with alterai. And that will be built by professionals software developers not by bipodors.

I would agree violently with you on that I think software development as a profession is going underway. I'm still answering your question. I have no crystal ball. I literally like cannot see. The but I agree. I think software development is going to know whether the one thing I learned more and more as I do this is I develop skills engineering software engineering. Especially when you get into sophisticated scale software engineering who's going nowhere as a profession at all. And for me and I prepare to take the other side of the deck and I learnt I genuinely am I think hallucination is going to work. I don't think hallucination problem is going away. He don't because I think it's inherent and so we had a new class of knowledge. But I think that's fine. I think that's absolutely fine. If you're software developers plus llms are incredibly powerful. And for me that's the main learning I've had is that it's the two things working together that really really give the power. I don't think there's going to be many citizen developers like you said for the sort of real business logic for anything sophisticated. It is always going to be a software engineering mindset.

I have lived, I'm old enough to live through. I didn't live in color and it wasn't quite bad but covalent was meant to take away some sort of element. Seemed meant to take away software development. 4GLs were going to take away suffering. It doesn't happen. This is no different. Never is very long time but.

It's definitely not going away in foreseeable future. Current tool set current LLMs current ways of building and even if we look one or two years forward. It's not going to like the it's clear that citizen developers can only go so far.

Section 10 — Audience Q1 — Is this all chaos?

Thank you. And full disclosure and when I use the okay I said aside we've probably got time for a couple of questions from the audience.

Yeah. Thank you. This is friends is helping us today. Thanks for doing the great discussion. Really enjoyed it. I just wanted to put on my contrarian hatch. It's like a popular view but I was listening to Bloomberg this morning and you were saying a bunch of like massive companies I just called cancelled or cloud code licenses like hooper and microsoft and yeah a lot of the kind of things that we were talking about there like a few like winter debt conferencing five years ago and you said you don't got any idea what's going on lost control lots of understanding random choices like doing stories. Like just sounds like total chaos. Do you think like this is really the way edge you're just going to go look in the future or like the values gives you there or it's where we're kind of you know it's going to be some implosion and like peers. And I'll just like preface that like if you go back like 10 years ago it feels very much like self driving cars. Like 2014 were like well stuff in cars like you know six months ago here you can drive to the airport back and it's perfect. But then all the edge cases and so on meant that you know there's still probably more taxi drivers in London today than there was back then. And you know all the self-driving car systems that are working today sort of the honorable humans you know kind of moving into kind of make sure that they don't get confused because you know they can only kind of work like 80% of the time. With software and then we're not going to get to have anything like reliable or retainable or sane to work with. Not really a question. More of a comment with the question.

No, it was a bit of a question. I will take that briefly. Microsoft code because they don't make money from it. They are radically pushing their engineers to use their co-pilot. There are also rebranding club coworkers, soft code. So it's a bit of a weird one with that microscope used to. Microsoft are still pushing their employees to use AI all the time 24/7. The other one I don't think they've actually got I think they've just they just opened up studying all of their AI budget in four months rather than 12. Like I was saying, I mean for me absolutely there is no such driving cars yes agree and that's what I was saying about you know self driving agents building all the software. It's co-driving. Very much co-driving but I think the quality is there. You know again I am hands off as after 30 years in the industry. I'm still building hands on software and the productivity gains are huge. The quality is there if you do harness engineering if you do not do good harmless engineering the quality is terrible. If you spend about as much time engineering your harness as you do on top table of the actual functional output, you will get really good quality stuff. Genuinely will. So I disagree that this is sort of fashioned. I think this is absolutely usable.

Section 11 — Audience Q2 — Cognitive load

Thank you. I think we've got time for more.

I think yeah. That's what you all been talking about like things get sped up like there is a lot more stuff right than I think when any team starts like I'm more successful like the most harsh resource becomes the human brain like and what do you do in your companies to like you know guide the cognitive resources of the organizations on the rights. It's nothing can be like what theory used to review what the page code should actually reach a human should it do that. Type. Of.

It's really difficult. What we see is that. Generally there is it goes both ways one unique to leverage the human brain but the second we need to protect it because there is a water but the nature of war with the eye is very different. Because normally you would think to a bit and then you would spend like two days cooling. Which is pretty much right it's not very demanding now you are thinking very hard and 10 minutes engagement goes away comes back and then you need to say hey this leads to a whole lot of like stress and cognitive load. And people are really throwing out if it's not managed correctly. On the other side you have a single person can generate massive amount of code and there's no need for a team. Without changes on our sort of back and forth between humans. So I don't have a clear answer for that but it's a really big topic. The work environment for developers will change dramatically right and we need to address this very explicitly. No unrest, no curacies because people like one of our customers said actually every few weeks they need to send a person for a week. Home. Just to relax because the pressure goes so fast but they'll blow this is amazing after a month of working like that which is still not in. Release pressure. To actually have multiple agents remaining yeah wonderful that sounds great that it would just destroy people very very quickly the context in those context switching working late hours because they want to see what they commit are coming now 10 p.m and 11 p.m. It's yeah and you cannot stop because like I want to see the result and so this is this is the sort of soft machine the variable ward which the yagis talked about it is it's addictive because you don't know I had afraid I have to wind this up and being given the T signal so thank you very much to the panel to Simon and to Samantha. Thank you very much.

---

(Following 15 seconds of applause, the recording continues into the next session — a separate talk by an Adobe principal scientist on building AI agents in the browser. That content is not part of this panel and is excluded from this skill.)