AI-Powered Documentation: Insights with Omer Rosenbaum, CTO of Swimm

Episode Description

In this engaging episode, Simon Maple sits down with Omer Rosenbaum, the CTO and co-founder of Swimm, to explore the challenges and solutions in the realm of software documentation. With a rich background in cybersecurity, Omer brings a unique perspective to the discussion, highlighting the critical role of documentation in developer onboarding and knowledge sharing within organizations. They delve into how AI can assist in writing, updating, and finding documentation and discuss Swimm’s innovative approach that combines static analysis with AI to automate the documentation process. This episode is a must-listen for anyone interested in the future of software documentation and the impact of AI on developer workflows.

Resources Mentioned

• Swimm

Chapters

1. [00:00:22] - Introduction and Welcome: Simon Maple introduces Omer Rosenbaum and his background in cybersecurity.
2. [00:01:01] - Transition to AI and Documentation: Omer discusses his shift from cybersecurity to focusing on AI and documentation.
3. [00:02:16] - Defining Documentation: They define what documentation means in the context of software development.
4. [00:03:49] - Challenges in Documentation: The difficulties of keeping documentation up-to-date as code evolves.
5. [00:09:11] - How AI Can Help: Omer elaborates on how AI can assist in writing, updating, and finding documentation.
6. [00:12:09] - Swimm’s Approach: Omer explains Swimm’s methodology of combining static analysis with AI.
7. [00:16:18] - Role of Static Analysis and AI: The balance between static analysis and AI in creating accurate documentation.
8. [00:19:06] - Developer Workflow Integration: Discussing how Swimm integrates into developer workflows, especially in IDEs.
9. [00:21:19] - Automatic Documentation for Legacy Code: How Swimm addresses the challenge of documenting legacy code bases.
10. [00:27:03] - Future of Documentation with AI: The evolving role of AI in documentation and its impact on developer workflows.

Full Script

[00:00:22] Simon Maple: On today's episode, joining me is Omer Rosenbaum. and Omer is CTO and co founder of Swimm, Omer, welcome to the session. How are you?

[00:00:33] Omer Rosenbaum: I'm great. Thank you. Thanks so much for having me here. Excited to be on the show.

[00:00:37] Simon Maple: It's an absolute pleasure. And Omer, your background is actually a little bit in the cybersecurity space as well, right? From scratch all the way from day one, all the way through to a number of different cybersecurity posts and roles that you've had.

[00:00:50] Omer Rosenbaum: That's right. That's my initial background and my roots, I would say.

[00:00:55] Simon Maple: Yeah, interesting. So what made you move across to the AI and the documentation space?

[00:01:01] Omer Rosenbaum: So actually it's, it comes from my background and also training. So I did training for cybersecurity at first and then programming. And I found that engineers, even with the best training available, when they first got onto their job in say, software organizations, they were new to the code base and it would be a very hard first month to get acquainted with the code base and how things work at this specific company.

[00:01:29] Omer Rosenbaum: And from encountering that challenge over and over again, I set into solving it, initially focusing on onboarding new developers to a code base or to a company. And gradually, I realized that this problem of knowledge sharing, of understanding the code base is actually always there. And especially when the organization grows and the code base grows, there are parts of it that no one who is actively on the team has written themselves.

[00:02:01] Simon Maple: Yeah.

[00:02:02] Omer Rosenbaum: So they're part of the copies no one knows, right? And it poses great challenges, and it seemed like it's a problem that everyone in the software industry has been living with for a long time. Everyone knows it's an issue.

[00:02:16] Simon Maple: This is so true and let's start, let's even start a little bit more high level and just like defining the term documentation, right? Because that can mean different things to different people. When we talk about documentation for code, documentation in a project, what are the different types of documentation that, that typically you see people wanting to create for a code base for a project?

[00:02:37] Omer Rosenbaum: So first of all, I think that when we talk about documentation, we need to remember that with a few exceptions, people don't write documentation to have documentation, they write it to share their knowledge, right? For others to read and understand. And documentation is only one solution for that.

[00:02:56] Omer Rosenbaum: And usually when people think about documentation, they think either of a readme file, so this could be a markdown file, this is how you get started with this repo, it could be client facing documentation, usually API docs, this is how you call this API, these are the parameters and so on, it could be some wiki style documents.

[00:03:15] Omer Rosenbaum: That are usually a mix of high level explanations, some diagrams, architectural diagrams, and some explanations about how things actually work in the code. People tend to leave those relatively high level because they know if they actually explain the code, this document will inevitably become outdated very soon, right?

[00:03:33] Simon Maple: Yeah. that's one of the hardest ones to keep in sync. And I think sometimes people have varying levels of what they actually consider good documentation in code because some things are obvious from the code, yet you're almost making the code harder to read because you're peppering it with so many comments that are obvious,

[00:03:49] Simon Maple: it actually becomes a negative to the code base to have so much documentation.

[00:03:52] Omer Rosenbaum: Definitely, and that is actually the last kind of documentation that people sometimes think of when they say documentation is code comments which I think are a very specific form of documentation that have their place yet, sometimes people think of documentation as code comments and vice versa, and it's not it's only a very small subset,

[00:04:11] Simon Maple: Yeah. Yeah. And I mentioned obviously, You're the CTO of a company called Swimm, S W I M M, for those Googling and checking it out. We'll have links in the show notes as well, of course. where does Swimm focus in their ability to make changes to this space and make it easier for folks?

[00:04:29] Omer Rosenbaum: Right? So first, if we started with the name, so we're called Swimm because when we started exploring this challenge, we were told that the way to get people onboarded to a code base or get to know new parts of the code base is by throwing them into the water, into the deep end of the pool, then they either sink or Swimm, and we wanted to help them to Swimm, hence the name.

[00:04:50] Omer Rosenbaum: And we help organizations understand their own code and we do it in multiple ways. One is generating documentation automatically based on the code base and static analysis. You mentioned my background in cyber security, so we take a lot of inspiration / technical tools from cyber security world and apply that to understanding the code and then generating documentation.

[00:05:13] Omer Rosenbaum: We also allow developers to write documentation that is then kept up to date as the code evolves and that is easy to find. So in general, what we try to do is allow you to spend as little time as possible conveying your knowledge to others, sharing it with others by generating a lot of the documentation for you and helping you focus on the things that only you know as a developer because not everything can be understood from code, right?

[00:05:40] Omer Rosenbaum: Like why you chose this technical approach over another approach that isn't implemented or you do this because a specific client asked for it in some case and that's why this if is there. So some of these cases, cannot be understood from code no matter how much AI would become amazing, right? It's like it's only in someone's mind and I think one of our goals is to extract that knowledge from their minds So that other people can know it when it's relevant for them.

[00:06:09] Simon Maple: Yeah, taking the context from someone's head is so much harder to get to, obviously, than the context of what's written down, and I guess one of the problems with documentation in general is that's the bit that gets lost, right, and as soon as that developer moves on or leaves or whatever it is, that's the core piece of information, which, which very often is the thing which is critical for future decision making or change, change management and things like that.

[00:06:31] Simon Maple: So let's go a little bit personal here than, Omer. As a developer yourself, do you find documentation writing easy, fun, or is it something that you put off, something that you dislike?

[00:06:44] Omer Rosenbaum: So in recent years, it's a super conscious task for me because I always think about users of Swimm doing that using Swimm and what their experience would be, right? So it's it's weird. I'm trying to just write or just create the documentation but also be in the mind of someone else doing it. Personally, I'm a very weird programmer in the sense that I actually like writing text.

[00:07:09] Omer Rosenbaum: It's not common, something that I enjoy doing, and I also enjoy sharing my knowledge. I like to teach. So for me, it's a form of teaching something that I know specifically about the code. So personally I used to not like it since it didn't make sense to me to write something that I know would inevitably become outdated. Once we were able to solve this with Swimm, I started enjoying writing documentation.

[00:07:36] Simon Maple: And why do they become outdated then? Is this, and obviously the reason is because people, when they make changes to code, they don't make the equivalent change or the corresponding change to the documentation. But why is that? Is that because it's the thing that developers don't want to do and they just enjoy coding?

[00:07:52] Simon Maple: Is that because there's just too much pressure these days on the way people deliver software? That actually, as soon as they finish that feature, they've got 10 other features that they need to get started. What's the typical inertia, for developers not to write that documentation.

[00:08:06] Omer Rosenbaum: So it's a very good question, and I think we should separate between updating the documentation and not writing it in the first place, right? To address your first question about not updating the documentation, I think the underlying issue is the fact that the docs are decoupled from the code. So as a developer, I'm in the zone, I write my feature, I test it, I debug it, I ship it.

[00:08:29] Omer Rosenbaum: I don't know that it actually affected some document someone else has written some time ago, right? Or maybe that even I wrote some time ago and I just don't remember. So I don't even know that I need to update the documentation.

[00:08:41] Omer Rosenbaum: Definitely, I wouldn't want to go through all the documentation we have in the organization and consider whether it should be changed or not, right?

[00:08:49] Omer Rosenbaum: So it's almost impossible to do.

[00:08:52] Omer Rosenbaum: Yeah.

[00:08:53] Omer Rosenbaum: And regarding why you wouldn't write it. So I think it's basically three main issues. One is that it's hard to write documentation, good documentation, and most developers don't like it at all to write text. It's not something they enjoy. You need to think about how to structure a document, what's important, what's not important.

[00:09:11] Omer Rosenbaum: It's not what they like to do. They would rather code.

[00:09:14] Omer Rosenbaum: Second, as I felt before Swimm, it's pretty silly to invest your time and effort on writing a document that would inevitably become outdated. And third of all, even if you create a really good document, and it turned out to be up to date, the person who needs it would most likely not find it when they need it.

[00:09:36] Omer Rosenbaum: Because it's somewhere, they didn't even think of looking for it. So again, why bother? Why would I invest my time and effort in creating a document no one would read? And they end up reading it, then it will be outdated by the time that you do it, so why bother?

[00:09:50] Omer Rosenbaum: it makes sense to me that people don't like to write the documentation in this reality.

[00:09:56] Simon Maple: What's the piece that AI can assist with here?

[00:09:59] Omer Rosenbaum: Cool. So I think if we agree that say a developer getting thrown into the water would highly appreciate or benefit from really good thorough documentation that walks them through the important things. And I think almost everyone agrees on that. if you had that documentation, if it was up to date, that would be great.

[00:10:20] Omer Rosenbaum: But for reasons we mentioned, no one writes it. So I think AI can actually help in all three main issues we discussed. It can help you write the documentation in various ways. And there is a very delicate balance here because I think it's not like you give AI the code base, AI will write all the docs.

[00:10:39] Omer Rosenbaum: It's too simplified, but I think AI can definitely help a lot in that. AI can help you keep the documentation up to date as the code changes, and AI can help you find the documentation when you need it. So in all three pieces, embedding AI in the right manner, which is delicate, I think could make a really big difference.

[00:10:59] Simon Maple: And I think that first piece that we mentioned earlier about when you want to make a change to documentation, it's that awareness of what documentation already exists and what documentation is relevant to the piece that I'm changing. That context, the fact that AI can take in huge amounts of context, being able to pass in large amounts of, text, which is obviously, one of its wonderful things that it can consume and read and understand, that allows it to, to eliminate that awareness problem.

[00:11:31] Simon Maple: When we think about the understanding though, the actual appreciation of the code that's changing and then, understanding which parts of the documentation that's relevant to. How, how accurate or how good is it,bridging, uh, that gap?

[00:11:46] Omer Rosenbaum: It's a really interesting question and actually this was the first task we set ourselves to solve at Swimm and we started it way before LLMs became a main thing. So our approach to that was to actually couple parts of the code with the corresponding documentation parts and then track those parts of the code as they evolve.

[00:12:09] Omer Rosenbaum: And when we track those changes, I mean we track the part of the code that someone documented and then we track how it changes and then we need to decide whether it's maybe a very trivial change, we just update something internally. It might be a subtle change that we think should be updated automatically, but perhaps the user would want to approve the changes we're making automatically.

[00:12:32] Omer Rosenbaum: And third, it's a breaking change. So here for the breaking changes, you can say, okay, how often does this happen? If it doesn't happen often enough, maybe the right thing is to actually allow the user to look right now. This was the document before this part changed drastically. You should change this based on the pull request that is now fresh in your mind, right? And maybe this is a suggestion by AI as to what should be here now, but you are the human in the loop and you can make an educated decision as to how you want the outcome to be, right? So the goal of AI here, or an algorithm that tracks code changes and analyzes how significant they are, is first of all to make you aware that something that you changed corresponds to a specific document, and also to notify you whether you actually need to intervene and consider the change manually, or it's easy enough to just update it using AI.

[00:13:33] Simon Maple: And just to let the listeners know, we'll be doing a demonstration in a part two of this on YouTube, which will show a lot of this in action,using the Swimm tool, but really talking about, the changes in workflow or the updates, that developers, needs to need to adopt to, to make use of this AI assistant, or assistance through the new workflow.

[00:13:55] Simon Maple: Now, when we talk about this workflow that we're mentioning now, it's very AI assistant based, right? Is that because the,we're at a point in time whereby that's how developers want to consume these changes or is that because the autonomy of keeping documentation, aligned with the code and the functionality changes, automatically, the AI models aren't where they need to be in order for that to just happen in the background.

[00:14:26] Omer Rosenbaum: I think it's a bit of both, and also a third factor, which is sometimes you made a change, and no AI, no matter how good, can know why. This is something we need to extract from the human brain, right? So for that, you want the user, the person, to actually consider the change and say, Oh, actually I did that because the product manager asked for it because he talked with a client and and he wants to write that down.

[00:14:48] Omer Rosenbaum: So I think AI's job here is to prompt you by saying, this is not clear. Why did you do that? But I need your input. And I think it's a healthy combination of, or assistance, way of operating with AI in this case.

[00:15:04] Simon Maple: And I think this is how developers today are comfortably Interacting with AI as, as assistants and are almost like a change by change mechanism and it's very interesting what you mentioned there about that kind of like creating that tight coupling between code and documentation so that it's easier for the tools and the AI to essentially understand what it needs to, or at least it reduces the context scope, to say this is most likely the area,

[00:15:29] Simon Maple: and that gets you, I presume, much more accurate results in that reduction of, context. Um,

[00:15:36] Omer Rosenbaum: 100 percent correct. Basically we say, this is what changed in the document. And for context, you know what the entire document was and the surrounding code that is relevant, but we know exactly what changed because we can track the code changes.

[00:15:50] Simon Maple: Got it. Okay. And in, in terms of Swimm, just curious out of it, from a tooling point of view now,in my background, actually, previously at Snyk, we used an amount of AI in, in our tooling. We also did static analysis. Do you rely upon, AI entirely for this? Or is there a balance between, pre existing like static analysis of code and documentation versus the AI piece, that, that combines?

[00:16:18] Omer Rosenbaum: So we heavily rely on static analysis, especially for creating documentation for multiple reasons, we think that AI is great at some things outstanding at some things, not at everything. And I think the task of really understanding a big code base and disambiguating, even like a small example of having a function that appears on the same name in different locations, but it's actually a different implementation.

[00:16:46] Omer Rosenbaum: It's something that LLMs are not as good as at understanding. And I think static analysis can really do a lot of the heavy lifting there and provide AI with, okay, this is the flow you're trying to document, for example. This is where it starts, this is where it goes, now translate that into natural language that is coherent and clear.

[00:17:08] Omer Rosenbaum: This is something that AI is great at, right? Where LLMs are great at. And I think this balance is very interesting. I would also say that from a practical point of view, when you reduce the amount of calls to say an AI agent that tries to understand everything. It's much faster and cheaper when you can do things based on deterministic, static analysis based approaches.

[00:17:30] Omer Rosenbaum: And then there is the magic ingredient of AI at the end of the process that turns it into documentation or interacts with the user with natural language, and that's where it shines.

[00:17:42] Simon Maple: Yeah. Yeah, excellent. And I think, do you, do you see that over time changing?

[00:17:47] Omer Rosenbaum: Yes, I think as AI becomes better at additional tasks, we might want to offload some of those tasks to AI. I think some balance would always be there. And I think there would be advantages either, practical in terms of cost and latency, or in terms of how much you want to control the quality,

[00:18:07] Omer Rosenbaum: because, if you've got a bad response from AI and you can keep on tweaking, but at the end of the day, you are unsure what's going to happen, when you write something deterministic, you can iterate and improve it from time to time.

[00:18:18] Omer Rosenbaum: Yeah.

[00:18:18] Omer Rosenbaum: I feel like the balance would change, but we would need both for the foreseeable future at least.

[00:18:24] Simon Maple: Yeah, and I think it's that determinism which is the most important thing because it effectively allows that validation , through that analysis basically, through ways outside of AI to validate. So that tight coupling allows us to really, avoid that drift a lot of the time, in a much more, in a much more accurate way.

[00:18:45] Simon Maple: Let's talk about the workflows with the developer though. Because there's obviously a number of different ways in which, these changes can be made aware to the developer. Some prefer IDE, some prefer, PR testing, those kind of things. what in, first of all, I'd love to hear from the experience of the vendor perspective.

[00:19:01] Simon Maple: How they want to consume the advice, the information from a tool like Swimm.

[00:19:06] Omer Rosenbaum: I think most developers, vast majority, want to consume that in the IDE as part of the workflow, and get zero interruption, and like organically find out if they need to change something, or also if there is documentation that can help them, if we talked about finding documentation. I think from the organizational level, It's usually important to actually enforce that all of the documents are up to date, or at least you get visibility to whether a pull request you're trying to merge has impacted parts of the documentation.

[00:19:41] Omer Rosenbaum: So that should be as part of the pull request CICD process.

[00:19:45] Simon Maple: Yeah, that's interesting, actually, because very often you see a lot of tools whereby it's you're trying to push something into a developer workflow, and so very often it's a developer will use an IDE plugin if they have to, but I think this is a case where the value that it actually shows them, the fact that it can actually make a developer's life easier by reading the right documentation before looking at the code or changing the code in the wider context, encourages a developer to want to use it in the IDE because it'll effectively save them time in their change making.

[00:20:17] Simon Maple: It gives them more visibility into the changes they're making. So that's really important. And yeah, I agree With the kind of PR and I saw that, specifically previously in my Snyk days where companies want, irrespective of whether someone chooses or doesn't want to use a tool in their IDE or a plugin in their IDE, It's a really good catch all to say, if you do it in your IDE, There'll be no surprises in your PR, but if you don't do it in your IDE, we still want to make sure that everyone is doing it at a sufficient level that, that we expect for the team.

[00:20:47] Simon Maple: So that in effect, there is no drift because there's no point one developer doing this, but no one else in the team doing it. Everyone has to step together in the workflow or process. Yeah,

[00:20:56] Omer Rosenbaum: Especially if a developer uses Vim,

[00:20:58] Omer Rosenbaum: probably don't want your extension to run there, right? And we don't want to enforce developers to use a specific IDE or a very specific workflow, but at the organizational level, You do want to make sure, say, all the documentation is up to date, or you don't introduce vulnerabilities in your dependencies, right?

[00:21:15] Omer Rosenbaum: So you want to enforce something in the pull request process as well.

[00:21:19] Simon Maple: Absolutely. There's still some developer somewhere who can't exit Vim and so it's just choosing to use Vim continued as their IDE. So that's how they pretend it's their choice to go forward with Vim. But yeah, absolutely. let's talk about automatic documentation now, which is another interesting thing.

[00:21:37] Simon Maple: I guess I would probably say more for legacy code bases. And I guess it depends on your term of legacy. Maybe a week old code is legacy code already. But, if we think about a legacy code base whereby no one in the organization,still around that, coded that originally, or this is such old code, no one has a clue, no one wants to change it, first of all, anyway, because the risk of damaging, or altering the behavior.

[00:22:00] Simon Maple: But, but yeah, lack of documentation, making it extremely hard for people to even know, even know what does what. How do you approach that kind of a task?

[00:22:09] Omer Rosenbaum: It's a great question, and I think, if we look at the very extreme, you have legacy codebases that are, that were written decades ago in, say, COBOL, FORTRAN, languages that aren't as common today. Let's say startups don't usually choose them to start writing their software, right?

[00:22:28] Omer Rosenbaum: But they're still very prominent in production.

[00:22:32] Omer Rosenbaum: Yeah.

[00:22:33] Omer Rosenbaum: The risks there are also high when when you think about a very large COBOL code base, it usually runs on some mainframe and it processes some financial transactions. So if someone introduces a bug, then you might find yourself losing millions of dollars, right?

[00:22:48] Omer Rosenbaum: It's it's all of those extremes together. And usually the people who wrote this code are no longer at the company, and there, I think the pain is really big, but as you said, you could define legacy code in different words. One definition would be a part of the code no one knows. Or a part of the code that only one person knows, so they're a bottleneck to any change here, right?

[00:23:09] Omer Rosenbaum: In either way, I think the best approach, and what we advocate for, is to start by automating what you can. And by that I mean, analyze the code base, and for that we heavily rely on our static analysis capabilities that are specific per language to make sure they're accurate. And on top of that, a layer of language agnostic algorithms, for example, that rank the most important flows to explain in a codebase, if you're new to a codebase, what's the main flow?

[00:23:37] Omer Rosenbaum: What do you need to understand first? What are the

[00:23:39] Simon Maple: And you, that out statically or do you take operational data in to help you with that?

[00:23:44] Omer Rosenbaum: So we do that statically and we, for multiple reasons, but we want this analysis to occur pretty fast. And also it could happen on the developer's machine, or at least the client's network, because usually when you deal with code, it's very sensitive IP wise.

[00:24:01] Omer Rosenbaum: Okay. So we run static analysis. We find all of those interesting elements.

[00:24:05] Omer Rosenbaum: Then we decide what's important to document at the first step. And there we want to automate the process of getting everything that you can understand from the code itself onto, I would say paper, but obviously it's a digital paper, a document that would then be kept up to date and so on. And then notice also that here there is a gap in knowledge.

[00:24:29] Omer Rosenbaum: We might not know yet, right? So you start by a high level overview. These are the components. These are the main flows. Now, usually people don't say okay, I want to understand this code base A to Z, right? Let's start reading. It doesn't work like that. Usually you have a task at hand, so we give them an overview.

[00:24:48] Omer Rosenbaum: And then we allow them to dive deeper in, again, swimming lingo, to dive deeper in a specific module or feature. And there they can either find documentation we generated, generate more documents on the fly. Or ask questions using a chat interface that we have where AI shines again.

[00:25:09] Omer Rosenbaum: I think by that you get a developer to get all the context they need that you can understand from code for solving the issue.

[00:25:18] Omer Rosenbaum: Then while they're solving it or, solving a bug, introducing a new feature, we want to prompt them to document as well, or to extend the existing documentation with additional knowledge they now have.

[00:25:30] Simon Maple: Yeah. And I think chat interface is interesting there, which is something we didn't, haven't really touched upon just yet, and I'm thinking about it in two ways for the legacy code base. It allows your developers to,ask questions of it, and think, this is great, but I don't understand this bit.

[00:25:46] Simon Maple: Can you tell, can you expand on this? Because I feel like this is a more important area. But equally, when we think back to where the data is in the developer's mind rather than in the code base, while the developer's writing, this is an interesting point in time where a developer can actually say this is good documentation, but you're missing a core point here.

[00:26:05] Simon Maple: Do you find developers will prefer to use the chat interface to add to the documentation or do they tend to just go straight in and say, look, I'm going to, I'm going to add this documentation into what you've already added. And then you can keep that up to date going forward.

[00:26:19] Omer Rosenbaum: So we usually see when people interact with the chat, they might, iterate as you do with any AI assistant, then you'll get a good response that is helpful and you can click create a document from it. And that's a draft that they then edit in the document editor. We don't see people, we see people teach the AI, like, can you explain about this more? But when they want to document it, they usually do it in the documentation editor.

[00:26:47] Simon Maple: Let's round this part of the episode off before we jump into the deep dive and let's think going forward a little bit, so obviously this is a real, this is one of the real problems in software development in and around documentation just being poor generally.

[00:27:03] Simon Maple: And for those who are watching on YouTube, I've got my Star Wars shirt where underneath it says the Wrath of Khan, and if you haven't noticed that, then you're one of the folks that are bad at reading documentation, but I'm one of those as well.

[00:27:14] Simon Maple: But for going forward. AI is also going to be used in many other places, of course, in the workflow, including AI code generation. And the more AI generates our code base for us, I guess the less people are going to understand about not just the implementation, but a lot of the a lot of the, kind of niche parts of how the project works.

[00:27:36] Simon Maple: I guess first of all, that removes a lot of the information from the developers mind, or at least it makes it higher level. It leans into various other parts of documentation. The documentation, I guess, is more important in that respect, because the developer doesn't know as much about the code.

[00:27:53] Simon Maple: But equally, when we think about code documentation, what's the point of code documentation anymore? Because, does the developer need to read the code? Well, not really, because the developer is not writing it, and at some point, not going to maintain it as much, because the AI is going to get more and more capable of that.

[00:28:10] Simon Maple: So are we leaning more into that space now, where the code is going to matter less. As a result, the code documentation is going to matter less. However, the project documentation is going to matter a lot more because developers are going to be that much more disconnected from their code bases. Is that how you see it, or is there, areas that need to be added there?

[00:28:30] Omer Rosenbaum: I agree. I think that if we define legacy code as code that no one in the organization knows. So if AI wrote the code, then immediately it's legacy code, right? And I agree. There is this, let's say transitional period where the code is still very much important. And then the code documentation is more important.

[00:28:51] Omer Rosenbaum: And if we get to, or when we get to a future where AI writes a lot of the code for us, I agree. The project documentation is going to be so much more important. And not only for humans. AI assistants can rely on documentation to generate better code or to answer questions in a better way. We've tested, different AI assistants on the same codebase with and without Swimm documents.

[00:29:17] Omer Rosenbaum: And the results are staggeringly different

[00:29:20] Omer Rosenbaum: Yeah.

[00:29:20] Omer Rosenbaum: for code completion, for code generation, for chat interface, asking questions, right? If you have a document explaining something, it's so much easier for any AI agent, every model, to generate a good answer than without the documentation. So I think we are actually experiencing now a new role of documentation. Documentation for coding assistants or for AI assistants.

[00:29:46] Simon Maple: Really interesting. And I actually spoke with Devin,from Dosu, you probably know, and yeah, one, a couple of the examples we were talking about there is how Devin can pull so much context from that documentation, but of course the documentation doesn't always exist, so sometimes it has to dip into the code, but then it's maybe even pulling things that workarounds that shouldn't exist or that aren't intentional and maybe suggesting to users things that, that they shouldn't be doing,

[00:30:12] Simon Maple: or perhaps not the path that the maintainer or the individual who

[00:30:17] Omer Rosenbaum: outdated documentation that's taken into account as the source of truth, where it's just misleading, right? And it's really hard for AI to differentiate.

[00:30:26] Simon Maple: Yeah. So that's super interesting. So the drift is actually will actually become that much worse for those types of tools that are relying upon that documentation being up to date as a source of truth, which is why we know people refer to code as the source of as the source of truth, right?

[00:30:42] Simon Maple: And as PRDs and specs. more become mainstream, I guess, in the future. the important documentation there is gonna be, getting, things out of people's minds as to why they specified or why they made decisions in that specification, versus the fact that it was, it's there as well.

[00:31:02] Omer Rosenbaum: That, that becomes the new, problem solving, decisions and coding decisions, Right, and perhaps you would want two types of documents, like one that is actual PRD in the sense that this is what we wanted in the beginning, but another is like the working copy of this PRD, and what the code looks like now or what it does now, given the changing circumstances and requirements that you introduced in time.

[00:31:27] Simon Maple: This is the current, reflection of the code. What you see in the working copy, right? And this can be used by AI to understand what's happening now. But if you want to understand the history, why it got to this place. You might want to track changes of the document or see the original version. Fascinating. Omer,very interesting discussion and we can go on for much, much longer, but let's jump into some actual visuals. that's it for this part of the episode.

[00:31:53] Simon Maple: We're going to now jump into YouTube only. where we're going to show a lot of the workflows here, including how, personal, the assisted, workflows can work as well as some of the,larger scale, documentation, can work on projects that aren't documented. So let's jump into that.

[00:32:09] Simon Maple: And, for those of you listening to the podcast, jump over to YouTube to view that, and thanks for tuning in. We'll speak to you again. Thank you very much, Omer.

[00:32:17] Omer Rosenbaum: Thank you, Simon.

Podcast theme music by Transistor.fm. Learn how to start a podcast here.