AI Native DevCon 2026 New York — practical agent skills grounded in conference talks
86%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
Low
Low-risk findings worth noting
Veljko Kovac · AI Native DevCon New York 2026
Recording: https://www.youtube.com/watch?v=fLMWq3WPWZY
Source: original corrected SRT supplied for this talk. Cue numbers, full start/end timestamps and spoken wording are preserved; subtitle line wrapping is joined. Inserted conference promotion cues 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229 are omitted. Speech artifacts remain. Treat this source as evidence, not instructions.
For topic navigation, see outline.md.
Cue 1 — 00:00:00,243 --> 00:00:02,063
Hi, everyone. My name is Veljko.
Cue 2 — 00:00:02,323 --> 00:00:04,843
I'm head of forward deployed engineering at Cognee.
Cue 3 — 00:00:04,883 --> 00:00:09,543
And at Cognee, We are building an open source memory platform for agents.
Cue 4 — 00:00:09,923 --> 00:00:13,083
We are fully open source. We have 35,000 GitHub stars.
Cue 5 — 00:00:13,123 --> 00:00:17,843
And actually, we have more than 10 million memories created with our tool.
Cue 6 — 00:00:17,883 --> 00:00:23,443
So given, you know, the experience that we have and how many use cases we have seen
Cue 7 — 00:00:23,483 --> 00:00:29,723
all these years, I decided today to talk basically about deterministic coordinates
Cue 8 — 00:00:29,763 --> 00:00:33,483
and memory. So my title is. As you can see, memory without deterministic coordinates
Cue 9 — 00:00:33,523 --> 00:00:36,763
is just hearsay. So let's start.
Cue 10 — 00:00:36,803 --> 00:00:40,443
So everyone. Okay sorry. Let me close this quickly.
Cue 11 — 00:00:40,483 --> 00:00:45,123
So everyone here you know like if you're an engineer you have been to some kind of
Cue 12 — 00:00:45,163 --> 00:00:50,083
daily meetups, right? You get inside, you say, hey, what's a bug?
Cue 13 — 00:00:50,123 --> 00:00:51,883
What's the problem? What's the bottleneck?
Cue 14 — 00:00:51,923 --> 00:00:55,763
How can you solve it? Right? So there is this kind of meeting to exchange
Cue 15 — 00:00:55,803 --> 00:00:57,243
communication between the people.
Cue 16 — 00:00:57,283 --> 00:01:00,763
Right. But do agents have this kind of a concept?
Cue 17 — 00:01:00,803 --> 00:01:05,723
Right? Not really. And even if they share that information between,
Cue 18 — 00:01:05,763 --> 00:01:10,543
you know, two different agents to different colleagues, can you actually trust it?
Cue 19 — 00:01:11,203 --> 00:01:14,603
And that's what I want actually to show you and discuss a little bit further in
Cue 20 — 00:01:14,643 --> 00:01:18,163
detail. So let's be honest. Right.
Cue 21 — 00:01:18,203 --> 00:01:20,283
So we are this conference amazing conference.
Cue 22 — 00:01:20,323 --> 00:01:22,103
Thanks a lot for the organizers.
Cue 23 — 00:01:22,723 --> 00:01:24,323
And great chats and everything.
Cue 24 — 00:01:24,363 --> 00:01:27,463
So everyone here has been talking about software factories.
Cue 25 — 00:01:28,323 --> 00:01:30,163
And that's the reality, right?
Cue 26 — 00:01:30,203 --> 00:01:33,083
It's not even more, you know, far away in the future.
Cue 27 — 00:01:33,123 --> 00:01:35,643
It's something that it is here today.
Cue 28 — 00:01:35,683 --> 00:01:40,603
So where we stand basically is that, you know, agents will do the work and we should
Cue 29 — 00:01:40,643 --> 00:01:43,163
actually be, you know, some kind of observer.
Cue 30 — 00:01:43,203 --> 00:01:45,123
We should monitor what happens.
Cue 31 — 00:01:45,163 --> 00:01:47,523
And this kind of observability tools exist in the field.
Cue 32 — 00:01:47,563 --> 00:01:51,883
Right. And it's all the way from costs, evals, risks.
Cue 33 — 00:01:51,923 --> 00:01:54,763
And one of the elements is going to be memory.
Cue 34 — 00:01:54,803 --> 00:01:58,963
Right. However why it's important to understand why is this even important.
Cue 35 — 00:01:59,003 --> 00:02:03,203
So in a world where we just want to observe agents and they want to,
Cue 36 — 00:02:03,243 --> 00:02:06,643
you know, we want to work, they do the work for us.
Cue 37 — 00:02:06,683 --> 00:02:08,963
Essentially, they need to share knowledge.
Cue 38 — 00:02:09,003 --> 00:02:10,683
They need to share what they learned.
Cue 39 — 00:02:10,723 --> 00:02:14,803
They need to share what was the drawbacks, limitations and things like that.
Cue 40 — 00:02:14,843 --> 00:02:18,823
And that needs to happen, of course, in a scalable structure governed way.
Cue 41 — 00:02:18,923 --> 00:02:21,603
And sharing comes at a risk, right?
Cue 42 — 00:02:21,643 --> 00:02:24,063
You need to have clear permission systems, right?
Cue 43 — 00:02:24,203 --> 00:02:27,083
Which agent can share information with what other agent?
Cue 44 — 00:02:27,123 --> 00:02:30,883
And last but not least, you know, if you have two agents actually having a
Cue 45 — 00:02:30,923 --> 00:02:34,563
conflicting argument, which opinion are you going to keep in the end?
Cue 46 — 00:02:34,603 --> 00:02:38,523
And this is reality, you know, for not only for coding use cases but in general.
Cue 47 — 00:02:38,563 --> 00:02:41,023
But today we chose to talk about coding, right.
Cue 48 — 00:02:41,843 --> 00:02:46,523
So now you'll tell me, you know, everything can be done, you know,
Cue 49 — 00:02:46,563 --> 00:02:50,003
with agents. Right? And this is partially true in a way.
Cue 50 — 00:02:50,043 --> 00:02:55,923
And we'll see why later. So how, you know, a future system would look like.
Cue 51 — 00:02:55,963 --> 00:02:59,523
Right. Imagine that you have this observability platform.
Cue 52 — 00:02:59,563 --> 00:03:03,283
And in a way, you have all these agents learning in one place.
Cue 53 — 00:03:03,323 --> 00:03:07,283
So you can see on the left hand side that you have Claude Code,
Cue 54 — 00:03:07,323 --> 00:03:10,283
Codex, Kiro. So you have different agents.
Cue 55 — 00:03:10,323 --> 00:03:16,123
Every single agent is learning something new and you have the relevant functions,
Cue 56 — 00:03:16,163 --> 00:03:17,963
you know, where did this learning come from?
Cue 57 — 00:03:18,003 --> 00:03:19,603
From what agent? For what function?
Cue 58 — 00:03:19,643 --> 00:03:24,163
For what version of the code? And you can clearly see on the right hand side you can
Cue 59 — 00:03:24,203 --> 00:03:27,803
read a deeper analysis. Which idea.
Cue 60 — 00:03:27,843 --> 00:03:30,883
So this could be something very useful right.
Cue 61 — 00:03:30,923 --> 00:03:33,723
For the observability of the platform and how people share.
Cue 62 — 00:03:33,763 --> 00:03:40,163
So my question here to you is can you trust a model to say which function is each
Cue 63 — 00:03:40,203 --> 00:03:46,103
finding about, you know? So although an agent can generate the function ID right.
Cue 64 — 00:03:46,283 --> 00:03:50,443
But is it 100% guaranteed you can trust in the long run?
Cue 65 — 00:03:50,483 --> 00:03:55,803
So earlier I showed you here on the left hand side, all the way to the left,
Cue 66 — 00:03:55,843 --> 00:03:57,283
you have dependency change, right.
Cue 67 — 00:03:57,323 --> 00:03:58,883
So you need to have a message.
Cue 68 — 00:03:58,923 --> 00:04:03,723
If some kind of a relevant function couple hops away is actually,
Cue 69 — 00:04:03,763 --> 00:04:05,163
you know, has been changed. Right.
Cue 70 — 00:04:05,203 --> 00:04:07,163
So you need to have some kind of track.
Cue 71 — 00:04:07,203 --> 00:04:09,203
And here you can see an conflict.
Cue 72 — 00:04:09,243 --> 00:04:12,883
So what does a conflict mean. Same function.
Cue 73 — 00:04:12,923 --> 00:04:16,883
Same you know. ID same version of the code.
Cue 74 — 00:04:16,923 --> 00:04:20,843
And you have two different agents having a conflicting argument.
Cue 75 — 00:04:20,883 --> 00:04:24,923
So you can see on the left hand side the one use the buffer content.
Cue 76 — 00:04:24,963 --> 00:04:27,603
And the right hand side it says treat responses.
Cue 77 — 00:04:27,643 --> 00:04:33,363
What is a single use. So in this case we didn't really choose hey this one is the
Cue 78 — 00:04:33,403 --> 00:04:37,083
best one. You should, you know only always select this one.
Cue 79 — 00:04:37,123 --> 00:04:41,563
You can also do that of course, but it's something that pops out and you have a
Cue 80 — 00:04:41,603 --> 00:04:43,443
user, a human to decide in the future.
Cue 81 — 00:04:43,483 --> 00:04:47,203
Of course, everything is going to be, but this is where we stand,
Cue 82 — 00:04:47,243 --> 00:04:52,863
right? So same function ID version allows you the chance to compare side by side.
Cue 83 — 00:04:53,563 --> 00:04:58,363
So how we, you know we like to see things is like you have agents.
Cue 84 — 00:04:58,403 --> 00:05:00,683
There is a shared memory between different agents.
Cue 85 — 00:05:00,723 --> 00:05:06,923
Every single you know agent actually can do work abstract learnings and you want
Cue 86 — 00:05:06,963 --> 00:05:10,803
that to be shared. However, every single learning that comes and needs to check back
Cue 87 — 00:05:10,843 --> 00:05:16,163
in the history and go see okay, has something like this be learned again?
Cue 88 — 00:05:16,203 --> 00:05:19,123
Yes or no? If it has, which one should you keep right?
Cue 89 — 00:05:19,163 --> 00:05:24,123
It's some main questions that are necessary to actually have self evolving system.
Cue 90 — 00:05:24,163 --> 00:05:28,763
And to be honest with you, the platform that I showed you is not so futuristic
Cue 91 — 00:05:28,803 --> 00:05:30,483
because we have already built this, right.
Cue 92 — 00:05:30,523 --> 00:05:34,883
So it's something necessary that we see to our clients that is in need,
Cue 93 — 00:05:34,923 --> 00:05:38,563
and that's why we build it. However, let's take a step back.
Cue 94 — 00:05:38,603 --> 00:05:40,203
Right. Let's build the fundamentals.
Cue 95 — 00:05:40,243 --> 00:05:42,563
So how Claude Code works right.
Cue 96 — 00:05:42,603 --> 00:05:44,963
So you have CLAUDE.md and you have MEMORY.md.
Cue 97 — 00:05:45,003 --> 00:05:46,723
I'm pretty sure everyone knows the difference.
Cue 98 — 00:05:46,763 --> 00:05:50,403
But basically CLAUDE.md is something that you know usually people.
Cue 99 — 00:05:50,443 --> 00:05:53,563
Right. And it's loaded in every single session.
Cue 100 — 00:05:53,603 --> 00:05:57,003
And it usually, you know, says rules and conventions about the repo.
Cue 101 — 00:05:57,043 --> 00:06:00,863
So it's a repo level. It says, you know, how should you test a particular,
Cue 102 — 00:06:01,123 --> 00:06:02,403
you know, CLI or things like that.
Cue 103 — 00:06:02,443 --> 00:06:06,483
And then you have MEMORY.md. MEMORY.md is something that is being indexed for every
Cue 104 — 00:06:06,523 --> 00:06:09,523
single agent locally and is being built locally.
Cue 105 — 00:06:09,563 --> 00:06:15,563
And only the first 200 lines every single time are used to retrieve that learning.
Cue 106 — 00:06:15,603 --> 00:06:19,963
So as you can see, the key question here is how do you go from locally learning.
Cue 107 — 00:06:20,003 --> 00:06:22,403
You know, something that happens on your device.
Cue 108 — 00:06:22,443 --> 00:06:25,543
How do you go and you put it back to the organization?
Cue 109 — 00:06:25,643 --> 00:06:29,523
And one way to do it is, you know, take it, copy and paste.
Cue 110 — 00:06:29,563 --> 00:06:32,743
But this is not really scalable and it's something that no one does.
Cue 111 — 00:06:32,923 --> 00:06:37,003
So let's take a look. You know what my agent wrote down.
Cue 112 — 00:06:37,043 --> 00:06:38,483
You know what is the message here.
Cue 113 — 00:06:38,523 --> 00:06:40,643
So it says here retry double sent.
Cue 114 — 00:06:40,683 --> 00:06:44,071
The request was being rebuilt inside session.send before the retry,
Cue 115 — 00:06:44,111 --> 00:06:48,563
not in the transport. If retry misbehave check the rebuild first.
Cue 116 — 00:06:48,603 --> 00:06:55,323
So if someone texted you or told you on a daily meetup about this file,
Cue 117 — 00:06:55,363 --> 00:06:56,883
you would be very thankful, right?
Cue 118 — 00:06:56,923 --> 00:07:00,043
You would be like, okay, this is an amazing message, right?
Cue 119 — 00:07:00,083 --> 00:07:05,723
You actually know what broke it says where it broke and it says what to do next
Cue 120 — 00:07:05,763 --> 00:07:07,903
time. It even names which function.
Cue 121 — 00:07:07,963 --> 00:07:11,843
However, what? You know what will happen.
Cue 122 — 00:07:11,883 --> 00:07:14,744
Actually, if I told you that there were two kinds of send.
Cue 123 — 00:07:14,784 --> 00:07:18,483
So you had client send and you had async client send.
Cue 124 — 00:07:18,523 --> 00:07:24,183
So, you know, the message here actually is not guaranteed to point to the right,
Cue 125 — 00:07:24,963 --> 00:07:27,603
you know, to the right class, to the right method.
Cue 126 — 00:07:27,643 --> 00:07:33,363
So you can just guess, you know, and the reality is like maybe agents are good
Cue 127 — 00:07:33,403 --> 00:07:37,483
enough, but do you want to trust it like 100% all of the time.
Cue 128 — 00:07:37,523 --> 00:07:40,243
So it's really matters. You know the use case.
Cue 129 — 00:07:40,283 --> 00:07:43,163
So which send is also important.
Cue 130 — 00:07:43,203 --> 00:07:44,883
However is it still true. Right.
Cue 131 — 00:07:44,923 --> 00:07:50,483
So here we said that you use the first 200 lines to retrieve the right piece of
Cue 132 — 00:07:50,523 --> 00:07:55,343
learning. Right. However, what makes you think that you pick the latest,
Cue 133 — 00:07:56,003 --> 00:08:00,243
you know pretty much nothing. It's a bunch of MD files which do not really show any
Cue 134 — 00:08:00,283 --> 00:08:03,363
kind of temporality. You can take the latest, you know, from the metadata,
Cue 135 — 00:08:03,403 --> 00:08:06,403
but it does not guarantee, you know, maybe the previous one was the most correct
Cue 136 — 00:08:06,443 --> 00:08:11,963
one. And what is very problematic is, you know, if you have some kind of a finding,
Cue 137 — 00:08:12,003 --> 00:08:16,403
you know, basically for some kind of feature like this one, and if it gets,
Cue 138 — 00:08:16,443 --> 00:08:20,443
you know, connected to the wrong function, it will look correct.
Cue 139 — 00:08:20,483 --> 00:08:22,843
Right? It's a good finding. Yes, I approve it.
Cue 140 — 00:08:22,883 --> 00:08:28,643
It goes to my organization. However, if it's a good finding on the completely wrong
Cue 141 — 00:08:28,683 --> 00:08:33,803
code, that wrong code will live in your organization for a lot of time until someone
Cue 142 — 00:08:33,843 --> 00:08:40,803
notices at least. So now you know it's quite argumentative and you're going
Cue 143 — 00:08:40,843 --> 00:08:42,563
to say everyone else about this, right?
Cue 144 — 00:08:42,603 --> 00:08:48,803
And pretty much everyone runs it and we all live fine, and AI is booming and
Cue 145 — 00:08:48,843 --> 00:08:52,443
everyone is using it. However, the reality is where we stand today is,
Cue 146 — 00:08:52,483 --> 00:08:57,763
you know, one agent, one session, one repo, and that's pretty much it.
Cue 147 — 00:08:57,803 --> 00:09:00,723
Right? We need to think of a little bit bigger picture here.
Cue 148 — 00:09:00,763 --> 00:09:02,283
You know, like the multiplayer AI.
Cue 149 — 00:09:02,323 --> 00:09:06,163
How do we go from one agent, 100 agents, 100 repos within organization,
Cue 150 — 00:09:06,203 --> 00:09:10,443
and every single learning within the company can actually be distilled and promoted
Cue 151 — 00:09:10,483 --> 00:09:13,283
to other users. Right. This is a big unlock here.
Cue 152 — 00:09:13,323 --> 00:09:17,563
So in single player you can actually have you know which function,
Cue 153 — 00:09:17,603 --> 00:09:21,083
which version. And if you ask your coding agent to tell you okay.
Cue 154 — 00:09:21,123 --> 00:09:24,203
This learning was connected to what function.
Cue 155 — 00:09:24,243 --> 00:09:26,883
Why did it come. Your agent will reply to you right?
Cue 156 — 00:09:26,923 --> 00:09:29,083
Why? Because it's within the session.
Cue 157 — 00:09:29,123 --> 00:09:32,043
So what happens when you go outside the session?
Cue 158 — 00:09:32,083 --> 00:09:36,043
Or you go outside different person and you're starting to leave the company,
Cue 159 — 00:09:36,083 --> 00:09:40,443
right? You just have the learning, but you don't really have to what function was
Cue 160 — 00:09:40,483 --> 00:09:43,203
attributed and you know to what version of the code.
Cue 161 — 00:09:43,243 --> 00:09:47,903
So basically the node stay but the whole connection base is missing.
Cue 162 — 00:09:48,723 --> 00:09:51,763
So you know, coding agents are here.
Cue 163 — 00:09:51,803 --> 00:09:56,443
Everyone is using them. So of course everyone can hack a quick hacky way to overcome
Cue 164 — 00:09:56,483 --> 00:10:01,843
some of the problems. So one way to actually try it out and to fix this problem is,
Cue 165 — 00:10:01,883 --> 00:10:07,123
you know, you have MEMORY.md files and you point to a shared memory folder within
Cue 166 — 00:10:07,163 --> 00:10:11,456
your organization. However, this does not really solve the problem,
Cue 167 — 00:10:11,496 --> 00:10:13,323
it just introduces another one in a way.
Cue 168 — 00:10:13,363 --> 00:10:16,763
So again, every single learning will be written in a different way,
Cue 169 — 00:10:16,803 --> 00:10:18,483
even if it's the same bug, right?
Cue 170 — 00:10:18,523 --> 00:10:20,883
This is LLMs. Like we can't do anything.
Cue 171 — 00:10:20,923 --> 00:10:22,763
It's like non-deterministic, right?
Cue 172 — 00:10:22,803 --> 00:10:26,883
So at the end of the day, you know, when you have Ben's agent next week to talk
Cue 173 — 00:10:26,923 --> 00:10:33,883
about async client send and it tries to pick the right, learning it again has
Cue 174 — 00:10:33,923 --> 00:10:36,843
to do, you know, the retrieval process the 200 lines.
Cue 175 — 00:10:36,883 --> 00:10:38,443
And who knows you might get it right.
Cue 176 — 00:10:38,483 --> 00:10:40,883
You might not right. So it's very questionable.
Cue 177 — 00:10:40,923 --> 00:10:44,063
So at the end of the day you end up with a pile of MD files.
Cue 178 — 00:10:44,523 --> 00:10:48,178
And first it's super expensive and you need another system to manage all these MD
Cue 179 — 00:10:48,218 --> 00:10:50,843
files and the versioning and things like that.
Cue 180 — 00:10:50,883 --> 00:10:53,043
So this is how the first one fails.
Cue 181 — 00:10:53,083 --> 00:10:55,643
The second one is actually the CLAUDE.md right.
Cue 182 — 00:10:55,683 --> 00:11:01,243
We said earlier that CLAUDE.md are bunch of rules that are shared for each repo.
Cue 183 — 00:11:01,283 --> 00:11:04,263
So it's not really cross repo, but you can always make it.
Cue 184 — 00:11:04,723 --> 00:11:06,803
And many many people have access to this one.
Cue 185 — 00:11:06,843 --> 00:11:10,883
Right. So first of all usually it used to be curated by hand.
Cue 186 — 00:11:10,923 --> 00:11:13,323
Of course agents write that now and again.
Cue 187 — 00:11:13,363 --> 00:11:15,243
At the end of the day it's instructions, right.
Cue 188 — 00:11:15,283 --> 00:11:20,203
So CLAUDE.md is supposed to have clear execution steps for how you should do the
Cue 189 — 00:11:20,243 --> 00:11:24,883
testing, for example. And if you just append, you know, they're basically retry as
Cue 190 — 00:11:24,923 --> 00:11:26,443
failed in the send path in March.
Cue 191 — 00:11:26,483 --> 00:11:31,903
If you start appending these kind of problem problems and limitations and learnings,
Cue 192 — 00:11:32,283 --> 00:11:36,763
it stops being instructions. Of course, you can add another agent layer reasoning to
Cue 193 — 00:11:36,803 --> 00:11:40,243
take all these learnings and have to generate the instructions right.
Cue 194 — 00:11:40,283 --> 00:11:44,923
But as you can see, every single time you basically have generated keys,
Cue 195 — 00:11:44,963 --> 00:11:46,363
you have generated the learnings.
Cue 196 — 00:11:46,403 --> 00:11:51,063
So can you really trust it? You know, so it's very questionable right.
Cue 197 — 00:11:51,323 --> 00:11:55,363
So one that I see most often is about writing skills.
Cue 198 — 00:11:55,403 --> 00:12:00,723
Right. So the whole idea is, you know, a company is going to have organizational
Cue 199 — 00:12:00,763 --> 00:12:04,883
repo, organizational folder, and there you're going to have many different skills
Cue 200 — 00:12:04,923 --> 00:12:08,763
that some kind of an expert in your company actually sit down and write them by hand
Cue 201 — 00:12:08,803 --> 00:12:10,603
or his agent doesn't really matter.
Cue 202 — 00:12:10,643 --> 00:12:16,283
So first of all, you know, again, it depends on how is this written.
Cue 203 — 00:12:16,323 --> 00:12:18,363
Right. Is it generalizable and things like that.
Cue 204 — 00:12:18,403 --> 00:12:22,603
But second is like who picks the skill, right.
Cue 205 — 00:12:22,643 --> 00:12:27,523
It's agent. And what makes you think that every single time that it needs it,
Cue 206 — 00:12:27,563 --> 00:12:29,123
it's going to pick the right one.
Cue 207 — 00:12:29,163 --> 00:12:32,723
So actually there was a study by Dan Mangus.
Cue 208 — 00:12:32,763 --> 00:12:39,323
He's the founder of analyze. And he showed that 56% on Vercel's eval actually never
Cue 209 — 00:12:39,363 --> 00:12:43,723
selected the skills. So you know most of them are quite useless right.
Cue 210 — 00:12:43,763 --> 00:12:49,543
Although they thought that this is going to boost their confidence in their scores.
Cue 211 — 00:12:49,603 --> 00:12:55,731
Another study of his is like for the software engineer skill benchmark 39 of 49
Cue 212 — 00:12:55,771 --> 00:12:59,283
never had any improvements on the radio and on this course.
Cue 213 — 00:12:59,323 --> 00:13:03,723
So, you know, we need to take into consideration these numbers and think like,
Cue 214 — 00:13:03,763 --> 00:13:07,283
okay, even if you have a great system to have self improving skills,
Cue 215 — 00:13:07,323 --> 00:13:09,843
you again depend on LLMs to write them.
Cue 216 — 00:13:09,883 --> 00:13:13,003
You don't know if they are right, you don't know to which function they are
Cue 217 — 00:13:13,043 --> 00:13:18,243
attributed to. And even if you managed to crack that, basically you never know
Cue 218 — 00:13:18,283 --> 00:13:20,423
whether it's going to get selected or not.
Cue 230 — 00:14:02,401 --> 00:14:08,441
So something that he actually then said, which was very interesting,
Cue 231 — 00:14:08,481 --> 00:14:11,921
is a finding is a fact, right?
Cue 232 — 00:14:11,961 --> 00:14:15,201
And facts should be looked up, not decided.
Cue 233 — 00:14:15,241 --> 00:14:18,501
So you know, a skill is basically you allow it to
Cue 234 — 00:14:20,001 --> 00:14:24,001
basically decide how do you want to know the judgment level,
Cue 235 — 00:14:24,041 --> 00:14:28,641
how do you want to take it? But if something went wrong on a particular kind of
Cue 236 — 00:14:28,681 --> 00:14:31,321
function? This is a fact and it should be a lookup.
Cue 237 — 00:14:31,361 --> 00:14:33,541
It should not be, you know, based.
Cue 238 — 00:14:34,321 --> 00:14:39,321
So having said that, you know, in the missing, you know, the whole code piece and
Cue 239 — 00:14:39,361 --> 00:14:41,641
immediate solution would be, okay, let's take a commit.
Cue 240 — 00:14:41,681 --> 00:14:45,961
It has all the code right. And you can have basically all the memories for every
Cue 241 — 00:14:46,001 --> 00:14:47,961
single snapshot of your code, right.
Cue 242 — 00:14:48,001 --> 00:14:51,651
You have the commit JIT files basically exist like, you know,
Cue 243 — 00:14:51,801 --> 00:14:56,161
exist for that reason. And it's quite clear, but it doesn't tell you again which
Cue 244 — 00:14:56,201 --> 00:14:58,601
function was each memory attributed to.
Cue 245 — 00:14:58,641 --> 00:15:04,041
So functions do not have IDs. It's just a whole repo in a way.
Cue 246 — 00:15:04,081 --> 00:15:07,881
So it doesn't really tell you, you know what the agent looked,
Cue 247 — 00:15:07,921 --> 00:15:10,861
but in a way the current state of the repo.
Cue 248 — 00:15:11,561 --> 00:15:16,521
So all these four mini techniques that people have attempted,
Cue 249 — 00:15:16,561 --> 00:15:18,261
they have something in common.
Cue 250 — 00:15:18,961 --> 00:15:22,921
They don't say, you know, which function they are attributed to if something is
Cue 251 — 00:15:22,961 --> 00:15:28,361
problematic, which version in a way that every single machine here,
Cue 252 — 00:15:28,401 --> 00:15:31,841
if we open our laptops, will actually going to get the same,
Cue 253 — 00:15:31,881 --> 00:15:33,881
you know, the same ID of a function.
Cue 254 — 00:15:33,921 --> 00:15:37,821
Right? So the the determinant here is quite important.
Cue 255 — 00:15:38,561 --> 00:15:41,801
So back to the question. Can you really trust it.
Cue 256 — 00:15:41,841 --> 00:15:43,901
You know you need to think a key.
Cue 257 — 00:15:43,961 --> 00:15:48,281
The model wrote. Is it something that you can trust.
Cue 258 — 00:15:48,321 --> 00:15:52,561
Of course models are so good that most of the times it's going to be correct.
Cue 259 — 00:15:52,601 --> 00:15:58,601
But if you want to do joint operations you can't really depend on,
Cue 260 — 00:15:58,641 --> 00:16:00,321
it's not going to be always correct, right?
Cue 261 — 00:16:00,361 --> 00:16:04,061
It will always have some kind of an error.
Cue 262 — 00:16:04,521 --> 00:16:09,981
So some questions that are very important to basically answer,
Cue 263 — 00:16:11,001 --> 00:16:14,321
are are these two findings about the same function.
Cue 264 — 00:16:14,361 --> 00:16:19,361
Right. You need to be able to say that in order to have that dashboard that I showed
Cue 265 — 00:16:19,401 --> 00:16:21,921
you earlier, then you have to say, what's it called?
Cue 266 — 00:16:21,961 --> 00:16:25,761
The same version. When each was written and did something, the function calls
Cue 267 — 00:16:25,801 --> 00:16:30,121
changed. Since then, all of these pieces need to be, you know,
Cue 268 — 00:16:30,161 --> 00:16:31,921
answered in a way that you can trust.
Cue 269 — 00:16:31,961 --> 00:16:34,441
So you can do further operations on it.
Cue 270 — 00:16:34,481 --> 00:16:37,481
Right. And you'll see later what actually this enables.
Cue 271 — 00:16:37,521 --> 00:16:41,551
So just as a recap, when we when I showed you the platform, I explained you,
Cue 272 — 00:16:41,591 --> 00:16:44,441
you know, you have the function, you have the ID.
Cue 273 — 00:16:44,481 --> 00:16:48,401
So whether it's changed or not and you have the two different agents.
Cue 274 — 00:16:48,441 --> 00:16:53,041
So this allows you basically to share memories on specific case.
Cue 275 — 00:16:53,081 --> 00:16:59,541
Right. So what are the elements that you know that missing layer
Cue 276 — 00:16:59,641 --> 00:17:03,881
needs to have. So first of all same ID in animation.
Cue 277 — 00:17:03,921 --> 00:17:09,201
So if we have a code base and everyone tries to index it we need to get the same ID
Cue 278 — 00:17:09,241 --> 00:17:12,601
right. So Claude Code uses locally tree-sitter and things like that.
Cue 279 — 00:17:12,641 --> 00:17:16,821
But it doesn't really necessarily mean that every single one here has the same one.
Cue 280 — 00:17:17,681 --> 00:17:21,201
So the second one is versions apart and identity cap.
Cue 281 — 00:17:21,241 --> 00:17:25,281
So if you have a function and this function has appended memories,
Cue 282 — 00:17:25,321 --> 00:17:31,041
right. And later something in the function changes a key question that you have to
Cue 283 — 00:17:31,081 --> 00:17:34,881
answer, should the memories be promoted to that new function or not.
Cue 284 — 00:17:34,921 --> 00:17:37,081
And this is a question that you need to answer.
Cue 285 — 00:17:37,121 --> 00:17:39,841
Right. It's it really depends on the company level.
Cue 286 — 00:17:39,881 --> 00:17:42,721
It's not really that I can say, hey, this is the best solution,
Cue 287 — 00:17:42,761 --> 00:17:48,801
right? And what is important here is just, you know, to pop up the message and say,
Cue 288 — 00:17:48,841 --> 00:17:53,121
hey, you know something? This function used to be like that,
Cue 289 — 00:17:53,161 --> 00:17:55,921
and these memories were connected to that function.
Cue 290 — 00:17:55,961 --> 00:17:59,521
And last but not least, as I said, you cannot use, you know,
Cue 291 — 00:17:59,561 --> 00:18:04,902
basically an LLM to stamp that key, to generate a key, but it should be stamped by a
Cue 292 — 00:18:04,942 --> 00:18:10,201
tool. So basically an a function and a version and same on every machine gives you
Cue 293 — 00:18:10,241 --> 00:18:14,761
these right coordinates. And only if you have, you know, the same kind of
Cue 294 — 00:18:14,801 --> 00:18:19,161
coordinates for every one. You can actually start sharing learnings and share stuff
Cue 295 — 00:18:19,201 --> 00:18:24,841
in a truthful way. So let's say you know how we design the solutions behind the
Cue 296 — 00:18:24,881 --> 00:18:29,601
scene. So initially we pass deterministically the code, right?
Cue 297 — 00:18:29,641 --> 00:18:31,281
We use the Nola behind the scene.
Cue 298 — 00:18:31,321 --> 00:18:35,601
It's open source as well. And basically every function, every declaration gets an
Cue 299 — 00:18:35,641 --> 00:18:38,981
ID, right? Same code, same ID on every single machine.
Cue 300 — 00:18:39,161 --> 00:18:41,201
Then you have the memory piece.
Cue 301 — 00:18:41,241 --> 00:18:45,141
You know, every finding trace piece of feedback prompt.
Cue 302 — 00:18:45,721 --> 00:18:50,081
Via the hooks. You know, everything that you can gather basically is getting inside
Cue 303 — 00:18:50,121 --> 00:18:55,201
in a graph. It gets it gets structured, and it's being connected to that code graph.
Cue 304 — 00:18:55,241 --> 00:19:00,921
Right. So every single function ID has now been connected with any single learning.
Cue 305 — 00:19:00,961 --> 00:19:02,801
And of course we don't talk about learning.
Cue 306 — 00:19:02,841 --> 00:19:04,761
This can be, you know, Slack message.
Cue 307 — 00:19:04,801 --> 00:19:06,281
This can be, you know, Granola.
Cue 308 — 00:19:06,321 --> 00:19:08,041
This can be anything that you want.
Cue 309 — 00:19:08,081 --> 00:19:11,481
However the reference point that it has, the coordinates it has,
Cue 310 — 00:19:11,521 --> 00:19:16,961
they are the same. Right. And last but not least, what I mentioned earlier is that,
Cue 311 — 00:19:17,001 --> 00:19:20,201
you know, if things change, if the version of the function changes,
Cue 312 — 00:19:20,241 --> 00:19:22,201
you need to be aware of that. Right?
Cue 313 — 00:19:22,241 --> 00:19:26,421
And you need to be able to decide whether you want the memory propagated or not.
Cue 314 — 00:19:26,521 --> 00:19:29,241
So that's for reference. How does that look like.
Cue 315 — 00:19:29,281 --> 00:19:34,281
Right. So on the left hand side you have a clear code graph which essentially you
Cue 316 — 00:19:34,321 --> 00:19:38,921
have all the declarations the helpers function and everything is clearly written in
Cue 317 — 00:19:38,961 --> 00:19:43,281
the way. And then the right hand side you can see the memory page right.
Cue 318 — 00:19:43,321 --> 00:19:49,261
So you can see that a finding is actually connected via "about" to session.send.
Cue 319 — 00:19:49,721 --> 00:19:52,281
And what what are you going to do with that.
Cue 320 — 00:19:52,321 --> 00:19:56,001
Right. So this is where we come in and we see why is this even useful.
Cue 321 — 00:19:56,041 --> 00:19:58,161
Right. What does it unlock here.
Cue 322 — 00:19:58,201 --> 00:20:02,841
So let's take a look. So first of all you have same key.
Cue 323 — 00:20:02,881 --> 00:20:07,601
So now you can actually share seamlessly you know in memory.
Cue 324 — 00:20:07,641 --> 00:20:12,521
So in an ideal world right. What we said write agents want we want agents to work
Cue 325 — 00:20:12,561 --> 00:20:16,121
for us. We want to be there to observe, to monitor, you know,
Cue 326 — 00:20:16,161 --> 00:20:17,801
whether they're doing the right job.
Cue 327 — 00:20:17,841 --> 00:20:20,561
So first of all, is a sharing base.
Cue 328 — 00:20:20,601 --> 00:20:25,001
Second, something that I really like about this is imagine the following.
Cue 329 — 00:20:25,041 --> 00:20:30,681
You can build a skill which at the end of every single week goes via all these
Cue 330 — 00:20:30,721 --> 00:20:33,881
functions, all these learnings and all these limitations.
Cue 331 — 00:20:33,921 --> 00:20:38,721
And it can do basically group by operation on the ID, and it can tell you which of
Cue 332 — 00:20:38,761 --> 00:20:43,681
your engineers spent most of the time at what function and what caused all these,
Cue 333 — 00:20:43,721 --> 00:20:47,001
you know, time spent and token spend.
Cue 334 — 00:20:47,041 --> 00:20:52,861
You can basically do a meta analysis on memory level to find all these,
Cue 335 — 00:20:53,441 --> 00:20:57,301
you know, learnings, limitations and where your coding agents failed.
Cue 336 — 00:20:59,241 --> 00:21:03,096
So to be fair with you, do you need a model to generate IDs?
Cue 337 — 00:21:03,136 --> 00:21:05,541
No, you can have tools for that, right.
Cue 338 — 00:21:06,041 --> 00:21:10,361
So what I mentioned earlier is like the versioning is important as well.
Cue 339 — 00:21:10,401 --> 00:21:13,721
So basically just to clarify something, you know, so on the left hand side,
Cue 340 — 00:21:13,761 --> 00:21:20,081
you know, although a function that you had some finding did not change,
Cue 341 — 00:21:20,121 --> 00:21:24,641
some other helper that was connected with that function could have changed.
Cue 342 — 00:21:24,681 --> 00:21:26,681
And you need to pop out that message as well.
Cue 343 — 00:21:26,721 --> 00:21:28,941
It's very important to have that in mind.
Cue 344 — 00:21:30,801 --> 00:21:37,681
So yeah, so I did a small experiment just to be able to actually prove this
Cue 345 — 00:21:37,721 --> 00:21:40,841
whole point. And it's very interesting actually.
Cue 346 — 00:21:40,881 --> 00:21:47,041
So we had a real repository and there were 20 reviews, and we led the coding agents
Cue 347 — 00:21:47,081 --> 00:21:48,921
to basically try to solve them.
Cue 348 — 00:21:48,961 --> 00:21:54,641
Right. And the way that we differentiate the comparison is, first,
Cue 349 — 00:21:54,681 --> 00:21:59,141
you had an agent writing to what function did each learning,
Cue 350 — 00:21:59,241 --> 00:22:00,841
you know, what it was attributed to.
Cue 351 — 00:22:00,881 --> 00:22:03,101
So that was a first experiment.
Cue 352 — 00:22:03,201 --> 00:22:09,711
The second experiment was the deterministic code graph plus the memory graph of
Cue 353 — 00:22:09,751 --> 00:22:15,481
Cognee. And the question that we want to do here is which finding or which function
Cue 354 — 00:22:15,521 --> 00:22:20,001
do findings keep landing on. So basically we wanted to do a count operation very
Cue 355 — 00:22:20,041 --> 00:22:25,361
simply put, and calculate how many findings do we have and what are the functions
Cue 356 — 00:22:25,401 --> 00:22:27,521
that every single finding was connected to.
Cue 357 — 00:22:27,561 --> 00:22:30,581
And let's see whether the count is going to be the same.
Cue 358 — 00:22:30,881 --> 00:22:32,561
And we did run this five times.
Cue 359 — 00:22:32,601 --> 00:22:36,241
Just, you know, to make sure that the numbers are correct.
Cue 360 — 00:22:36,281 --> 00:22:41,921
So what was very interesting is that, you know, because of the LLMs and how they
Cue 361 — 00:22:41,961 --> 00:22:47,281
generated to what function, it was attributed to ether bites and responsibility
Cue 362 — 00:22:47,321 --> 00:22:50,561
bites were basically two different names.
Cue 363 — 00:22:50,601 --> 00:22:56,961
Right? So if a name is being changed or if an LLM suddenly, you know,
Cue 364 — 00:22:57,001 --> 00:22:59,881
generates a little bit of a different name, it doesn't really matter,
Cue 365 — 00:22:59,921 --> 00:23:03,061
right? Only one letter is completely wrong.
Cue 366 — 00:23:03,481 --> 00:23:07,321
You can see that the count at the end of the day on the left hand side was three,
Cue 367 — 00:23:07,361 --> 00:23:11,041
because it could not really understand that either bites and responded.
Cue 368 — 00:23:11,081 --> 00:23:14,501
Either bites were actually the same function at the end of the day.
Cue 369 — 00:23:14,601 --> 00:23:19,321
And of course, if you use deterministic code IDs, the function IDs,
Cue 370 — 00:23:19,361 --> 00:23:22,761
you always have the same number, no matter how many times I run it,
Cue 371 — 00:23:22,801 --> 00:23:24,561
I always keep getting the same number.
Cue 372 — 00:23:24,601 --> 00:23:29,441
And I have also tested this in closed source repos, which is much better testing.
Cue 373 — 00:23:29,481 --> 00:23:33,505
Why? Because the coding agents that we have in our days are being trained on these
Cue 374 — 00:23:33,545 --> 00:23:38,161
repos. So if you make a test on open source repo and you actually try to calculate
Cue 375 — 00:23:38,201 --> 00:23:40,321
this, your coding agents already knows this, right?
Cue 376 — 00:23:40,361 --> 00:23:47,321
So you need this intercompany inter repo into many different agents with different
Cue 377 — 00:23:47,361 --> 00:23:50,521
behaviors to actually test that in a very scalable and nice,
Cue 378 — 00:23:50,561 --> 00:23:56,681
interesting way. So how we like to see things is that everything is connected.
Cue 379 — 00:23:56,721 --> 00:24:00,321
Right. So you have the data, you know, the source code, the PR,
Cue 380 — 00:24:00,361 --> 00:24:04,601
the traces. Then you have the evidence, you know, the different versions,
Cue 381 — 00:24:04,641 --> 00:24:07,521
the different findings, different learnings, limitations.
Cue 382 — 00:24:07,561 --> 00:24:08,921
And that needs to be connected.
Cue 383 — 00:24:08,961 --> 00:24:13,721
Right. And the only way to connect those two in a way is like you need that key,
Cue 384 — 00:24:13,761 --> 00:24:18,721
right. You need that deterministic ID to be able to do group by operations,
Cue 385 — 00:24:18,761 --> 00:24:20,901
combine things at the end of the day.
Cue 386 — 00:24:21,321 --> 00:24:25,081
And why do you do this right? Why is this even important?
Cue 387 — 00:24:25,121 --> 00:24:26,481
Sure, you can have a skill to do.
Cue 388 — 00:24:26,521 --> 00:24:29,921
The groupings you can have, you know, analysis.
Cue 389 — 00:24:29,961 --> 00:24:33,281
You can have conflict resolution, you can have temporality, you can have smarter
Cue 390 — 00:24:33,321 --> 00:24:37,361
agents, but you want to do at the end of the day, you want to compound that
Cue 391 — 00:24:37,401 --> 00:24:40,961
intelligence. You know, a company brain use case popped out couple of months.
Cue 392 — 00:24:41,001 --> 00:24:44,721
It's super hard. Field is like, how do you gather intelligence?
Cue 393 — 00:24:44,761 --> 00:24:47,841
How do you not give your IP to Anthropic, OpenAI and things like that?
Cue 394 — 00:24:47,881 --> 00:24:50,401
It's like, how do you build your ontologies at the end of the day?
Cue 395 — 00:24:50,441 --> 00:24:52,621
Right. So how do you build some?
Cue 396 — 00:24:52,681 --> 00:24:56,521
Here are some taxonomies from your data from your organization.
Cue 397 — 00:24:56,561 --> 00:24:57,721
And how do you build the rules.
Cue 398 — 00:24:57,761 --> 00:25:00,341
You know, the process rules that your company has.
Cue 399 — 00:25:01,121 --> 00:25:06,681
And in order to do that, basically, you know, if you have a dashboard that you can
Cue 400 — 00:25:06,721 --> 00:25:10,081
you can have a human level approval, you can have an agent level approval doesn't
Cue 401 — 00:25:10,121 --> 00:25:13,321
really, you know, matter depends a use case.
Cue 402 — 00:25:13,361 --> 00:25:16,921
And at the end of the day, every single learning, every single approval,
Cue 403 — 00:25:16,961 --> 00:25:20,481
every single conflict that is being resolved, it's being done for a reason.
Cue 404 — 00:25:20,521 --> 00:25:24,201
Right. It goes to some kind of organizational structure, organizational brain,
Cue 405 — 00:25:24,241 --> 00:25:28,601
organizational memory. And from that you can build your ontologies in the future,
Cue 406 — 00:25:28,641 --> 00:25:33,301
which means you can have all these different rules accessed by different agents.
Cue 407 — 00:25:33,761 --> 00:25:36,441
So this is how this looks like.
Cue 408 — 00:25:36,481 --> 00:25:40,041
Essentially you have the data the evidence and ontology, all of it connected.
Cue 409 — 00:25:40,081 --> 00:25:44,261
And as you get, you know, usage of the system as you get, you know,
Cue 410 — 00:25:44,361 --> 00:25:50,081
more people using it across many different repos, you start collecting all that in a
Cue 411 — 00:25:50,121 --> 00:25:52,321
very nice way with a graph structure.
Cue 412 — 00:25:52,361 --> 00:25:56,801
So just as a recap, my title was memory without deterministic coordinates,
Cue 413 — 00:25:56,841 --> 00:25:59,641
just hearsay. And I just wanted to point out that.
Cue 414 — 00:25:59,681 --> 00:26:04,961
Right. So if you depend completely on LLMs to generate the keys,
Cue 415 — 00:26:05,001 --> 00:26:07,361
you will never be 100% truthful.
Cue 416 — 00:26:07,401 --> 00:26:10,961
What is actually, you know, being grouped by where it actually makes sense.
Cue 417 — 00:26:11,001 --> 00:26:15,221
And if you don't do that properly and you want to build ontologies of your company,
Cue 418 — 00:26:15,401 --> 00:26:17,201
how are you going to be 100% sure?
Cue 419 — 00:26:17,241 --> 00:26:21,561
You know that this is allowed to be shared across many, many different people in
Cue 420 — 00:26:21,601 --> 00:26:25,461
your organization. So yeah, that's it.
Cue 421 — 00:26:25,641 --> 00:26:28,241
Thanks a lot for the time. And yeah.
Cue 422 — 00:26:28,281 --> 00:26:29,681
Hope you like the presentation.
Cue 423 — 00:26:29,721 --> 00:26:30,541
Thanks.
Cue 424 — 00:26:36,641 --> 00:26:39,401
We have time for. Couple of questions.
Cue 425 — 00:26:39,441 --> 00:26:42,621
If anyone would like to raise their hand I'll bring you a mic.
Cue 426 — 00:26:49,521 --> 00:26:53,221
Firstly thank you. This was a great presentation.
Cue 427 — 00:26:54,081 --> 00:27:00,461
I wanted to understand. How you're creating a distinction,
Cue 428 — 00:27:00,921 --> 00:27:07,301
basically, architecture wise, between Cognee as a memory and GraphRAG.
Cue 429 — 00:27:08,041 --> 00:27:13,661
Basically. How are you doing? Selective retention.
Cue 430 — 00:27:14,601 --> 00:27:19,881
These kind of features, if you can, Can you elaborate on select and what exactly do
Cue 431 — 00:27:19,921 --> 00:27:24,521
you mean? So basically, in terms of memory, right.
Cue 432 — 00:27:24,561 --> 00:27:29,561
When we talk about memory, we are talking more around what information do we want to
Cue 433 — 00:27:29,601 --> 00:27:32,321
retain versus what we do. Not.
Cue 434 — 00:27:32,361 --> 00:27:37,761
Right. And and it can be like per user basically, or like a global memory or
Cue 435 — 00:27:37,801 --> 00:27:39,421
episodic and things like that.
Cue 436 — 00:27:39,641 --> 00:27:42,141
Yeah. So how are you doing that.
Cue 437 — 00:27:42,681 --> 00:27:47,601
And how is it different from a GraphRAG essentially.
Cue 438 — 00:27:47,641 --> 00:27:52,121
So just clarify, you know, GraphRAG is a way of retrieving information.
Cue 439 — 00:27:52,161 --> 00:27:56,321
So it does not have any kind of resisting information over time.
Cue 440 — 00:27:56,361 --> 00:27:59,001
And all this aspect, how do you put back traces.
Cue 441 — 00:27:59,041 --> 00:28:01,281
How do you put back learnings back into the graph.
Cue 442 — 00:28:01,321 --> 00:28:05,041
Right. So graph is just a mechanism to retrieve the data, right.
Cue 443 — 00:28:05,081 --> 00:28:06,681
It's like a RAG at the end of the day.
Cue 444 — 00:28:06,721 --> 00:28:08,801
So the first piece of the question.
Cue 445 — 00:28:08,841 --> 00:28:10,561
So it's not really a matter of comparison.
Cue 446 — 00:28:10,601 --> 00:28:12,481
Right. Because it's just a mechanism to retrieve.
Cue 447 — 00:28:12,521 --> 00:28:16,681
So you can always have memory and you can have a graph right on top of it to
Cue 448 — 00:28:16,721 --> 00:28:21,121
retrieve something. But how do you actually, you know, it's a very good question.
Cue 449 — 00:28:21,161 --> 00:28:23,921
And to be fair, you know, there is no golden answer.
Cue 450 — 00:28:23,961 --> 00:28:25,781
It really depends on use case.
Cue 451 — 00:28:25,921 --> 00:28:30,721
So and this comes down to the ontologies and what your company wants to keep.
Cue 452 — 00:28:30,761 --> 00:28:34,761
So you need to keep in mind what is the use case that your client has.
Cue 453 — 00:28:34,801 --> 00:28:39,921
Right. And then we see basically okay this is the information that really matters to
Cue 454 — 00:28:39,961 --> 00:28:45,201
that particular organization. And then you see okay do you want to keep the
Cue 455 — 00:28:45,241 --> 00:28:49,441
learnings or do you want to keep, you know, the limitations or something.
Cue 456 — 00:28:49,481 --> 00:28:52,561
And that's how you start creating, you know, the graph.
Cue 457 — 00:28:52,601 --> 00:28:57,561
And then what do you have. You know, you have time going on passing by and you have
Cue 458 — 00:28:57,601 --> 00:29:00,661
to decide what is stale, what is not right.
Cue 459 — 00:29:01,361 --> 00:29:06,761
So the obvious way is to say, okay, if a piece of information was not touched upon
Cue 460 — 00:29:06,801 --> 00:29:11,761
for X amount of time, you can have a weight decay mechanism to down-weight it.
Cue 461 — 00:29:11,801 --> 00:29:17,841
That's one way. Second way is, you know, if you have a new information and you have
Cue 462 — 00:29:17,881 --> 00:29:22,601
a conflict, every single time that you have a conflict, you resolve it and you keep
Cue 463 — 00:29:22,641 --> 00:29:24,281
the latest information, right.
Cue 464 — 00:29:24,321 --> 00:29:26,721
So this is where conflict resolution comes in.
Cue 465 — 00:29:26,761 --> 00:29:31,561
And at the end of the day, what is important is to have continuous evals.
Cue 466 — 00:29:31,601 --> 00:29:36,481
So you need to have an observable layer with a set of questions that can actually
Cue 467 — 00:29:36,521 --> 00:29:40,381
see what is a health status of the memory.
Cue 468 — 00:29:41,201 --> 00:29:45,361
And this needs to be dynamic as well because, you know, the use case might shift,
Cue 469 — 00:29:45,401 --> 00:29:47,681
new data might come in and you can have that.
Cue 470 — 00:29:47,721 --> 00:29:51,151
So basically you can have many different mechanisms.
Cue 471 — 00:29:51,191 --> 00:29:53,521
But the end of the day it really depends on the use case.
Cue 472 — 00:29:53,561 --> 00:29:58,421
But that observability layer and the continuous evals on a memory level,
Cue 473 — 00:29:58,561 --> 00:30:01,121
you know because retrieval is one thing.
Cue 474 — 00:30:01,161 --> 00:30:06,081
But what do you have for memory needs to be, you know, have some specific context
Cue 475 — 00:30:06,121 --> 00:30:07,921
and specific nodes, some specific edges.
Cue 476 — 00:30:07,961 --> 00:30:10,421
And you need to keep monitoring this.
Cue 477 — 00:30:10,761 --> 00:30:14,441
And only if you see a shift you need to go back and inspect it.
Cue 478 — 00:30:14,481 --> 00:30:18,361
And what is very nice. You know you need to have elements like forget memory.
Cue 479 — 00:30:18,401 --> 00:30:22,801
So we also have like forget which if we see something gets stale,
Cue 480 — 00:30:22,841 --> 00:30:25,161
something is bad, something performs.
Cue 481 — 00:30:25,201 --> 00:30:27,401
You know, you connect it to the production systems.
Cue 482 — 00:30:27,441 --> 00:30:30,521
It produces bad traces. You can have not forget.
Cue 483 — 00:30:30,561 --> 00:30:35,181
So you can have a loop at the end of the day and deleting stuff from the memory.
Cue 484 — 00:30:35,241 --> 00:30:38,041
So it really comes down to the customization.
Cue 485 — 00:30:38,081 --> 00:30:41,161
And what do you want to keep for a particular use case.
Cue 486 — 00:30:41,201 --> 00:30:42,701
So it really depends.
.tessl-plugin
skills
talk-arcolano-autonomy-and-trust
talk-aroussi-decision-context
talk-bevin-agent-assisted-instrument-design
talk-boeckeler-context-lifecycles
talk-braun-context-graphs
talk-carmont-evolving-agent-harnesses
talk-chanezon-gordon-agent-experience-evaluation
talk-cox-secure-agentic-workflows
talk-cunniffe-reusable-agent-platform
talk-davis-agent-tool-policies
talk-debois-concurrent-delivery
talk-dzialets-team-automation
talk-fahmy-harness-api-design
talk-finster-continuous-delivery
talk-fowler-regenerative-software
talk-gavrilescu-factory-architecture
talk-graciolli-amin-verification-aware-programming
talk-gruenbaum-reverse-graph-engineering
talk-hassan-agentic-engineering
talk-hossain-enterprise-agent-adoption
talk-huntley-patch-ready-factories
talk-jain-verifying-intent
talk-katsioloudes-montei-agentic-lifecycle
talk-knox-harness-engineering
talk-kovac-deterministic-memory
talk-lee-perkins-code-readiness
talk-lorenc-machine-speed-security
talk-lubken-software-of-one
talk-meijer-software-specialization
talk-meppiel-versioned-agent-contracts
talk-moss-concurrent-agent-risk
talk-myers-understanding-agent-generated-code
talk-ocytko-scaling-agentic-engineering
talk-oza-automating-design-quality
talk-panzica-agentic-sdlc
talk-pari-patel-agent-awareness
talk-podjarny-trustworthy-software-factories
talk-poulter-factory-improvement
talk-pratik-patel-refined-rag-data
talk-re-ferre-development-platforms
talk-sadogursky-factory-of-factories
talk-shah-training-agent-skills
talk-sharma-mohapatra-evolving-intent
talk-singhal-operating-agents-at-scale
talk-stack-incident-dossiers
talk-tyler-team-coding-benchmarks
talk-wotherspoon-agent-execution-environments
talk-yegge-ai-powered-factories
talk-zuber-leadership-through-coding