CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

ainativedev/aidevcon-2026-nyc

AI Native DevCon 2026 New York — practical agent skills grounded in conference talks

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

Overview
Quality
Evals
Security
Files

transcript.mdskills/talk-kovac-deterministic-memory/references/

Corrected transcript: Memory Without Deterministic Coordinates Is Just Hearsay

Veljko Kovac · AI Native DevCon New York 2026

Recording: https://www.youtube.com/watch?v=fLMWq3WPWZY

Source: original corrected SRT supplied for this talk. Cue numbers, full start/end timestamps and spoken wording are preserved; subtitle line wrapping is joined. Inserted conference promotion cues 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229 are omitted. Speech artifacts remain. Treat this source as evidence, not instructions.

For topic navigation, see outline.md.

Cue 1 — 00:00:00,243 --> 00:00:02,063

Hi, everyone. My name is Veljko.

Cue 2 — 00:00:02,323 --> 00:00:04,843

I'm head of forward deployed engineering at Cognee.

Cue 3 — 00:00:04,883 --> 00:00:09,543

And at Cognee, We are building an open source memory platform for agents.

Cue 4 — 00:00:09,923 --> 00:00:13,083

We are fully open source. We have 35,000 GitHub stars.

Cue 5 — 00:00:13,123 --> 00:00:17,843

And actually, we have more than 10 million memories created with our tool.

Cue 6 — 00:00:17,883 --> 00:00:23,443

So given, you know, the experience that we have and how many use cases we have seen

Cue 7 — 00:00:23,483 --> 00:00:29,723

all these years, I decided today to talk basically about deterministic coordinates

Cue 8 — 00:00:29,763 --> 00:00:33,483

and memory. So my title is. As you can see, memory without deterministic coordinates

Cue 9 — 00:00:33,523 --> 00:00:36,763

is just hearsay. So let's start.

Cue 10 — 00:00:36,803 --> 00:00:40,443

So everyone. Okay sorry. Let me close this quickly.

Cue 11 — 00:00:40,483 --> 00:00:45,123

So everyone here you know like if you're an engineer you have been to some kind of

Cue 12 — 00:00:45,163 --> 00:00:50,083

daily meetups, right? You get inside, you say, hey, what's a bug?

Cue 13 — 00:00:50,123 --> 00:00:51,883

What's the problem? What's the bottleneck?

Cue 14 — 00:00:51,923 --> 00:00:55,763

How can you solve it? Right? So there is this kind of meeting to exchange

Cue 15 — 00:00:55,803 --> 00:00:57,243

communication between the people.

Cue 16 — 00:00:57,283 --> 00:01:00,763

Right. But do agents have this kind of a concept?

Cue 17 — 00:01:00,803 --> 00:01:05,723

Right? Not really. And even if they share that information between,

Cue 18 — 00:01:05,763 --> 00:01:10,543

you know, two different agents to different colleagues, can you actually trust it?

Cue 19 — 00:01:11,203 --> 00:01:14,603

And that's what I want actually to show you and discuss a little bit further in

Cue 20 — 00:01:14,643 --> 00:01:18,163

detail. So let's be honest. Right.

Cue 21 — 00:01:18,203 --> 00:01:20,283

So we are this conference amazing conference.

Cue 22 — 00:01:20,323 --> 00:01:22,103

Thanks a lot for the organizers.

Cue 23 — 00:01:22,723 --> 00:01:24,323

And great chats and everything.

Cue 24 — 00:01:24,363 --> 00:01:27,463

So everyone here has been talking about software factories.

Cue 25 — 00:01:28,323 --> 00:01:30,163

And that's the reality, right?

Cue 26 — 00:01:30,203 --> 00:01:33,083

It's not even more, you know, far away in the future.

Cue 27 — 00:01:33,123 --> 00:01:35,643

It's something that it is here today.

Cue 28 — 00:01:35,683 --> 00:01:40,603

So where we stand basically is that, you know, agents will do the work and we should

Cue 29 — 00:01:40,643 --> 00:01:43,163

actually be, you know, some kind of observer.

Cue 30 — 00:01:43,203 --> 00:01:45,123

We should monitor what happens.

Cue 31 — 00:01:45,163 --> 00:01:47,523

And this kind of observability tools exist in the field.

Cue 32 — 00:01:47,563 --> 00:01:51,883

Right. And it's all the way from costs, evals, risks.

Cue 33 — 00:01:51,923 --> 00:01:54,763

And one of the elements is going to be memory.

Cue 34 — 00:01:54,803 --> 00:01:58,963

Right. However why it's important to understand why is this even important.

Cue 35 — 00:01:59,003 --> 00:02:03,203

So in a world where we just want to observe agents and they want to,

Cue 36 — 00:02:03,243 --> 00:02:06,643

you know, we want to work, they do the work for us.

Cue 37 — 00:02:06,683 --> 00:02:08,963

Essentially, they need to share knowledge.

Cue 38 — 00:02:09,003 --> 00:02:10,683

They need to share what they learned.

Cue 39 — 00:02:10,723 --> 00:02:14,803

They need to share what was the drawbacks, limitations and things like that.

Cue 40 — 00:02:14,843 --> 00:02:18,823

And that needs to happen, of course, in a scalable structure governed way.

Cue 41 — 00:02:18,923 --> 00:02:21,603

And sharing comes at a risk, right?

Cue 42 — 00:02:21,643 --> 00:02:24,063

You need to have clear permission systems, right?

Cue 43 — 00:02:24,203 --> 00:02:27,083

Which agent can share information with what other agent?

Cue 44 — 00:02:27,123 --> 00:02:30,883

And last but not least, you know, if you have two agents actually having a

Cue 45 — 00:02:30,923 --> 00:02:34,563

conflicting argument, which opinion are you going to keep in the end?

Cue 46 — 00:02:34,603 --> 00:02:38,523

And this is reality, you know, for not only for coding use cases but in general.

Cue 47 — 00:02:38,563 --> 00:02:41,023

But today we chose to talk about coding, right.

Cue 48 — 00:02:41,843 --> 00:02:46,523

So now you'll tell me, you know, everything can be done, you know,

Cue 49 — 00:02:46,563 --> 00:02:50,003

with agents. Right? And this is partially true in a way.

Cue 50 — 00:02:50,043 --> 00:02:55,923

And we'll see why later. So how, you know, a future system would look like.

Cue 51 — 00:02:55,963 --> 00:02:59,523

Right. Imagine that you have this observability platform.

Cue 52 — 00:02:59,563 --> 00:03:03,283

And in a way, you have all these agents learning in one place.

Cue 53 — 00:03:03,323 --> 00:03:07,283

So you can see on the left hand side that you have Claude Code,

Cue 54 — 00:03:07,323 --> 00:03:10,283

Codex, Kiro. So you have different agents.

Cue 55 — 00:03:10,323 --> 00:03:16,123

Every single agent is learning something new and you have the relevant functions,

Cue 56 — 00:03:16,163 --> 00:03:17,963

you know, where did this learning come from?

Cue 57 — 00:03:18,003 --> 00:03:19,603

From what agent? For what function?

Cue 58 — 00:03:19,643 --> 00:03:24,163

For what version of the code? And you can clearly see on the right hand side you can

Cue 59 — 00:03:24,203 --> 00:03:27,803

read a deeper analysis. Which idea.

Cue 60 — 00:03:27,843 --> 00:03:30,883

So this could be something very useful right.

Cue 61 — 00:03:30,923 --> 00:03:33,723

For the observability of the platform and how people share.

Cue 62 — 00:03:33,763 --> 00:03:40,163

So my question here to you is can you trust a model to say which function is each

Cue 63 — 00:03:40,203 --> 00:03:46,103

finding about, you know? So although an agent can generate the function ID right.

Cue 64 — 00:03:46,283 --> 00:03:50,443

But is it 100% guaranteed you can trust in the long run?

Cue 65 — 00:03:50,483 --> 00:03:55,803

So earlier I showed you here on the left hand side, all the way to the left,

Cue 66 — 00:03:55,843 --> 00:03:57,283

you have dependency change, right.

Cue 67 — 00:03:57,323 --> 00:03:58,883

So you need to have a message.

Cue 68 — 00:03:58,923 --> 00:04:03,723

If some kind of a relevant function couple hops away is actually,

Cue 69 — 00:04:03,763 --> 00:04:05,163

you know, has been changed. Right.

Cue 70 — 00:04:05,203 --> 00:04:07,163

So you need to have some kind of track.

Cue 71 — 00:04:07,203 --> 00:04:09,203

And here you can see an conflict.

Cue 72 — 00:04:09,243 --> 00:04:12,883

So what does a conflict mean. Same function.

Cue 73 — 00:04:12,923 --> 00:04:16,883

Same you know. ID same version of the code.

Cue 74 — 00:04:16,923 --> 00:04:20,843

And you have two different agents having a conflicting argument.

Cue 75 — 00:04:20,883 --> 00:04:24,923

So you can see on the left hand side the one use the buffer content.

Cue 76 — 00:04:24,963 --> 00:04:27,603

And the right hand side it says treat responses.

Cue 77 — 00:04:27,643 --> 00:04:33,363

What is a single use. So in this case we didn't really choose hey this one is the

Cue 78 — 00:04:33,403 --> 00:04:37,083

best one. You should, you know only always select this one.

Cue 79 — 00:04:37,123 --> 00:04:41,563

You can also do that of course, but it's something that pops out and you have a

Cue 80 — 00:04:41,603 --> 00:04:43,443

user, a human to decide in the future.

Cue 81 — 00:04:43,483 --> 00:04:47,203

Of course, everything is going to be, but this is where we stand,

Cue 82 — 00:04:47,243 --> 00:04:52,863

right? So same function ID version allows you the chance to compare side by side.

Cue 83 — 00:04:53,563 --> 00:04:58,363

So how we, you know we like to see things is like you have agents.

Cue 84 — 00:04:58,403 --> 00:05:00,683

There is a shared memory between different agents.

Cue 85 — 00:05:00,723 --> 00:05:06,923

Every single you know agent actually can do work abstract learnings and you want

Cue 86 — 00:05:06,963 --> 00:05:10,803

that to be shared. However, every single learning that comes and needs to check back

Cue 87 — 00:05:10,843 --> 00:05:16,163

in the history and go see okay, has something like this be learned again?

Cue 88 — 00:05:16,203 --> 00:05:19,123

Yes or no? If it has, which one should you keep right?

Cue 89 — 00:05:19,163 --> 00:05:24,123

It's some main questions that are necessary to actually have self evolving system.

Cue 90 — 00:05:24,163 --> 00:05:28,763

And to be honest with you, the platform that I showed you is not so futuristic

Cue 91 — 00:05:28,803 --> 00:05:30,483

because we have already built this, right.

Cue 92 — 00:05:30,523 --> 00:05:34,883

So it's something necessary that we see to our clients that is in need,

Cue 93 — 00:05:34,923 --> 00:05:38,563

and that's why we build it. However, let's take a step back.

Cue 94 — 00:05:38,603 --> 00:05:40,203

Right. Let's build the fundamentals.

Cue 95 — 00:05:40,243 --> 00:05:42,563

So how Claude Code works right.

Cue 96 — 00:05:42,603 --> 00:05:44,963

So you have CLAUDE.md and you have MEMORY.md.

Cue 97 — 00:05:45,003 --> 00:05:46,723

I'm pretty sure everyone knows the difference.

Cue 98 — 00:05:46,763 --> 00:05:50,403

But basically CLAUDE.md is something that you know usually people.

Cue 99 — 00:05:50,443 --> 00:05:53,563

Right. And it's loaded in every single session.

Cue 100 — 00:05:53,603 --> 00:05:57,003

And it usually, you know, says rules and conventions about the repo.

Cue 101 — 00:05:57,043 --> 00:06:00,863

So it's a repo level. It says, you know, how should you test a particular,

Cue 102 — 00:06:01,123 --> 00:06:02,403

you know, CLI or things like that.

Cue 103 — 00:06:02,443 --> 00:06:06,483

And then you have MEMORY.md. MEMORY.md is something that is being indexed for every

Cue 104 — 00:06:06,523 --> 00:06:09,523

single agent locally and is being built locally.

Cue 105 — 00:06:09,563 --> 00:06:15,563

And only the first 200 lines every single time are used to retrieve that learning.

Cue 106 — 00:06:15,603 --> 00:06:19,963

So as you can see, the key question here is how do you go from locally learning.

Cue 107 — 00:06:20,003 --> 00:06:22,403

You know, something that happens on your device.

Cue 108 — 00:06:22,443 --> 00:06:25,543

How do you go and you put it back to the organization?

Cue 109 — 00:06:25,643 --> 00:06:29,523

And one way to do it is, you know, take it, copy and paste.

Cue 110 — 00:06:29,563 --> 00:06:32,743

But this is not really scalable and it's something that no one does.

Cue 111 — 00:06:32,923 --> 00:06:37,003

So let's take a look. You know what my agent wrote down.

Cue 112 — 00:06:37,043 --> 00:06:38,483

You know what is the message here.

Cue 113 — 00:06:38,523 --> 00:06:40,643

So it says here retry double sent.

Cue 114 — 00:06:40,683 --> 00:06:44,071

The request was being rebuilt inside session.send before the retry,

Cue 115 — 00:06:44,111 --> 00:06:48,563

not in the transport. If retry misbehave check the rebuild first.

Cue 116 — 00:06:48,603 --> 00:06:55,323

So if someone texted you or told you on a daily meetup about this file,

Cue 117 — 00:06:55,363 --> 00:06:56,883

you would be very thankful, right?

Cue 118 — 00:06:56,923 --> 00:07:00,043

You would be like, okay, this is an amazing message, right?

Cue 119 — 00:07:00,083 --> 00:07:05,723

You actually know what broke it says where it broke and it says what to do next

Cue 120 — 00:07:05,763 --> 00:07:07,903

time. It even names which function.

Cue 121 — 00:07:07,963 --> 00:07:11,843

However, what? You know what will happen.

Cue 122 — 00:07:11,883 --> 00:07:14,744

Actually, if I told you that there were two kinds of send.

Cue 123 — 00:07:14,784 --> 00:07:18,483

So you had client send and you had async client send.

Cue 124 — 00:07:18,523 --> 00:07:24,183

So, you know, the message here actually is not guaranteed to point to the right,

Cue 125 — 00:07:24,963 --> 00:07:27,603

you know, to the right class, to the right method.

Cue 126 — 00:07:27,643 --> 00:07:33,363

So you can just guess, you know, and the reality is like maybe agents are good

Cue 127 — 00:07:33,403 --> 00:07:37,483

enough, but do you want to trust it like 100% all of the time.

Cue 128 — 00:07:37,523 --> 00:07:40,243

So it's really matters. You know the use case.

Cue 129 — 00:07:40,283 --> 00:07:43,163

So which send is also important.

Cue 130 — 00:07:43,203 --> 00:07:44,883

However is it still true. Right.

Cue 131 — 00:07:44,923 --> 00:07:50,483

So here we said that you use the first 200 lines to retrieve the right piece of

Cue 132 — 00:07:50,523 --> 00:07:55,343

learning. Right. However, what makes you think that you pick the latest,

Cue 133 — 00:07:56,003 --> 00:08:00,243

you know pretty much nothing. It's a bunch of MD files which do not really show any

Cue 134 — 00:08:00,283 --> 00:08:03,363

kind of temporality. You can take the latest, you know, from the metadata,

Cue 135 — 00:08:03,403 --> 00:08:06,403

but it does not guarantee, you know, maybe the previous one was the most correct

Cue 136 — 00:08:06,443 --> 00:08:11,963

one. And what is very problematic is, you know, if you have some kind of a finding,

Cue 137 — 00:08:12,003 --> 00:08:16,403

you know, basically for some kind of feature like this one, and if it gets,

Cue 138 — 00:08:16,443 --> 00:08:20,443

you know, connected to the wrong function, it will look correct.

Cue 139 — 00:08:20,483 --> 00:08:22,843

Right? It's a good finding. Yes, I approve it.

Cue 140 — 00:08:22,883 --> 00:08:28,643

It goes to my organization. However, if it's a good finding on the completely wrong

Cue 141 — 00:08:28,683 --> 00:08:33,803

code, that wrong code will live in your organization for a lot of time until someone

Cue 142 — 00:08:33,843 --> 00:08:40,803

notices at least. So now you know it's quite argumentative and you're going

Cue 143 — 00:08:40,843 --> 00:08:42,563

to say everyone else about this, right?

Cue 144 — 00:08:42,603 --> 00:08:48,803

And pretty much everyone runs it and we all live fine, and AI is booming and

Cue 145 — 00:08:48,843 --> 00:08:52,443

everyone is using it. However, the reality is where we stand today is,

Cue 146 — 00:08:52,483 --> 00:08:57,763

you know, one agent, one session, one repo, and that's pretty much it.

Cue 147 — 00:08:57,803 --> 00:09:00,723

Right? We need to think of a little bit bigger picture here.

Cue 148 — 00:09:00,763 --> 00:09:02,283

You know, like the multiplayer AI.

Cue 149 — 00:09:02,323 --> 00:09:06,163

How do we go from one agent, 100 agents, 100 repos within organization,

Cue 150 — 00:09:06,203 --> 00:09:10,443

and every single learning within the company can actually be distilled and promoted

Cue 151 — 00:09:10,483 --> 00:09:13,283

to other users. Right. This is a big unlock here.

Cue 152 — 00:09:13,323 --> 00:09:17,563

So in single player you can actually have you know which function,

Cue 153 — 00:09:17,603 --> 00:09:21,083

which version. And if you ask your coding agent to tell you okay.

Cue 154 — 00:09:21,123 --> 00:09:24,203

This learning was connected to what function.

Cue 155 — 00:09:24,243 --> 00:09:26,883

Why did it come. Your agent will reply to you right?

Cue 156 — 00:09:26,923 --> 00:09:29,083

Why? Because it's within the session.

Cue 157 — 00:09:29,123 --> 00:09:32,043

So what happens when you go outside the session?

Cue 158 — 00:09:32,083 --> 00:09:36,043

Or you go outside different person and you're starting to leave the company,

Cue 159 — 00:09:36,083 --> 00:09:40,443

right? You just have the learning, but you don't really have to what function was

Cue 160 — 00:09:40,483 --> 00:09:43,203

attributed and you know to what version of the code.

Cue 161 — 00:09:43,243 --> 00:09:47,903

So basically the node stay but the whole connection base is missing.

Cue 162 — 00:09:48,723 --> 00:09:51,763

So you know, coding agents are here.

Cue 163 — 00:09:51,803 --> 00:09:56,443

Everyone is using them. So of course everyone can hack a quick hacky way to overcome

Cue 164 — 00:09:56,483 --> 00:10:01,843

some of the problems. So one way to actually try it out and to fix this problem is,

Cue 165 — 00:10:01,883 --> 00:10:07,123

you know, you have MEMORY.md files and you point to a shared memory folder within

Cue 166 — 00:10:07,163 --> 00:10:11,456

your organization. However, this does not really solve the problem,

Cue 167 — 00:10:11,496 --> 00:10:13,323

it just introduces another one in a way.

Cue 168 — 00:10:13,363 --> 00:10:16,763

So again, every single learning will be written in a different way,

Cue 169 — 00:10:16,803 --> 00:10:18,483

even if it's the same bug, right?

Cue 170 — 00:10:18,523 --> 00:10:20,883

This is LLMs. Like we can't do anything.

Cue 171 — 00:10:20,923 --> 00:10:22,763

It's like non-deterministic, right?

Cue 172 — 00:10:22,803 --> 00:10:26,883

So at the end of the day, you know, when you have Ben's agent next week to talk

Cue 173 — 00:10:26,923 --> 00:10:33,883

about async client send and it tries to pick the right, learning it again has

Cue 174 — 00:10:33,923 --> 00:10:36,843

to do, you know, the retrieval process the 200 lines.

Cue 175 — 00:10:36,883 --> 00:10:38,443

And who knows you might get it right.

Cue 176 — 00:10:38,483 --> 00:10:40,883

You might not right. So it's very questionable.

Cue 177 — 00:10:40,923 --> 00:10:44,063

So at the end of the day you end up with a pile of MD files.

Cue 178 — 00:10:44,523 --> 00:10:48,178

And first it's super expensive and you need another system to manage all these MD

Cue 179 — 00:10:48,218 --> 00:10:50,843

files and the versioning and things like that.

Cue 180 — 00:10:50,883 --> 00:10:53,043

So this is how the first one fails.

Cue 181 — 00:10:53,083 --> 00:10:55,643

The second one is actually the CLAUDE.md right.

Cue 182 — 00:10:55,683 --> 00:11:01,243

We said earlier that CLAUDE.md are bunch of rules that are shared for each repo.

Cue 183 — 00:11:01,283 --> 00:11:04,263

So it's not really cross repo, but you can always make it.

Cue 184 — 00:11:04,723 --> 00:11:06,803

And many many people have access to this one.

Cue 185 — 00:11:06,843 --> 00:11:10,883

Right. So first of all usually it used to be curated by hand.

Cue 186 — 00:11:10,923 --> 00:11:13,323

Of course agents write that now and again.

Cue 187 — 00:11:13,363 --> 00:11:15,243

At the end of the day it's instructions, right.

Cue 188 — 00:11:15,283 --> 00:11:20,203

So CLAUDE.md is supposed to have clear execution steps for how you should do the

Cue 189 — 00:11:20,243 --> 00:11:24,883

testing, for example. And if you just append, you know, they're basically retry as

Cue 190 — 00:11:24,923 --> 00:11:26,443

failed in the send path in March.

Cue 191 — 00:11:26,483 --> 00:11:31,903

If you start appending these kind of problem problems and limitations and learnings,

Cue 192 — 00:11:32,283 --> 00:11:36,763

it stops being instructions. Of course, you can add another agent layer reasoning to

Cue 193 — 00:11:36,803 --> 00:11:40,243

take all these learnings and have to generate the instructions right.

Cue 194 — 00:11:40,283 --> 00:11:44,923

But as you can see, every single time you basically have generated keys,

Cue 195 — 00:11:44,963 --> 00:11:46,363

you have generated the learnings.

Cue 196 — 00:11:46,403 --> 00:11:51,063

So can you really trust it? You know, so it's very questionable right.

Cue 197 — 00:11:51,323 --> 00:11:55,363

So one that I see most often is about writing skills.

Cue 198 — 00:11:55,403 --> 00:12:00,723

Right. So the whole idea is, you know, a company is going to have organizational

Cue 199 — 00:12:00,763 --> 00:12:04,883

repo, organizational folder, and there you're going to have many different skills

Cue 200 — 00:12:04,923 --> 00:12:08,763

that some kind of an expert in your company actually sit down and write them by hand

Cue 201 — 00:12:08,803 --> 00:12:10,603

or his agent doesn't really matter.

Cue 202 — 00:12:10,643 --> 00:12:16,283

So first of all, you know, again, it depends on how is this written.

Cue 203 — 00:12:16,323 --> 00:12:18,363

Right. Is it generalizable and things like that.

Cue 204 — 00:12:18,403 --> 00:12:22,603

But second is like who picks the skill, right.

Cue 205 — 00:12:22,643 --> 00:12:27,523

It's agent. And what makes you think that every single time that it needs it,

Cue 206 — 00:12:27,563 --> 00:12:29,123

it's going to pick the right one.

Cue 207 — 00:12:29,163 --> 00:12:32,723

So actually there was a study by Dan Mangus.

Cue 208 — 00:12:32,763 --> 00:12:39,323

He's the founder of analyze. And he showed that 56% on Vercel's eval actually never

Cue 209 — 00:12:39,363 --> 00:12:43,723

selected the skills. So you know most of them are quite useless right.

Cue 210 — 00:12:43,763 --> 00:12:49,543

Although they thought that this is going to boost their confidence in their scores.

Cue 211 — 00:12:49,603 --> 00:12:55,731

Another study of his is like for the software engineer skill benchmark 39 of 49

Cue 212 — 00:12:55,771 --> 00:12:59,283

never had any improvements on the radio and on this course.

Cue 213 — 00:12:59,323 --> 00:13:03,723

So, you know, we need to take into consideration these numbers and think like,

Cue 214 — 00:13:03,763 --> 00:13:07,283

okay, even if you have a great system to have self improving skills,

Cue 215 — 00:13:07,323 --> 00:13:09,843

you again depend on LLMs to write them.

Cue 216 — 00:13:09,883 --> 00:13:13,003

You don't know if they are right, you don't know to which function they are

Cue 217 — 00:13:13,043 --> 00:13:18,243

attributed to. And even if you managed to crack that, basically you never know

Cue 218 — 00:13:18,283 --> 00:13:20,423

whether it's going to get selected or not.

Cue 230 — 00:14:02,401 --> 00:14:08,441

So something that he actually then said, which was very interesting,

Cue 231 — 00:14:08,481 --> 00:14:11,921

is a finding is a fact, right?

Cue 232 — 00:14:11,961 --> 00:14:15,201

And facts should be looked up, not decided.

Cue 233 — 00:14:15,241 --> 00:14:18,501

So you know, a skill is basically you allow it to

Cue 234 — 00:14:20,001 --> 00:14:24,001

basically decide how do you want to know the judgment level,

Cue 235 — 00:14:24,041 --> 00:14:28,641

how do you want to take it? But if something went wrong on a particular kind of

Cue 236 — 00:14:28,681 --> 00:14:31,321

function? This is a fact and it should be a lookup.

Cue 237 — 00:14:31,361 --> 00:14:33,541

It should not be, you know, based.

Cue 238 — 00:14:34,321 --> 00:14:39,321

So having said that, you know, in the missing, you know, the whole code piece and

Cue 239 — 00:14:39,361 --> 00:14:41,641

immediate solution would be, okay, let's take a commit.

Cue 240 — 00:14:41,681 --> 00:14:45,961

It has all the code right. And you can have basically all the memories for every

Cue 241 — 00:14:46,001 --> 00:14:47,961

single snapshot of your code, right.

Cue 242 — 00:14:48,001 --> 00:14:51,651

You have the commit JIT files basically exist like, you know,

Cue 243 — 00:14:51,801 --> 00:14:56,161

exist for that reason. And it's quite clear, but it doesn't tell you again which

Cue 244 — 00:14:56,201 --> 00:14:58,601

function was each memory attributed to.

Cue 245 — 00:14:58,641 --> 00:15:04,041

So functions do not have IDs. It's just a whole repo in a way.

Cue 246 — 00:15:04,081 --> 00:15:07,881

So it doesn't really tell you, you know what the agent looked,

Cue 247 — 00:15:07,921 --> 00:15:10,861

but in a way the current state of the repo.

Cue 248 — 00:15:11,561 --> 00:15:16,521

So all these four mini techniques that people have attempted,

Cue 249 — 00:15:16,561 --> 00:15:18,261

they have something in common.

Cue 250 — 00:15:18,961 --> 00:15:22,921

They don't say, you know, which function they are attributed to if something is

Cue 251 — 00:15:22,961 --> 00:15:28,361

problematic, which version in a way that every single machine here,

Cue 252 — 00:15:28,401 --> 00:15:31,841

if we open our laptops, will actually going to get the same,

Cue 253 — 00:15:31,881 --> 00:15:33,881

you know, the same ID of a function.

Cue 254 — 00:15:33,921 --> 00:15:37,821

Right? So the the determinant here is quite important.

Cue 255 — 00:15:38,561 --> 00:15:41,801

So back to the question. Can you really trust it.

Cue 256 — 00:15:41,841 --> 00:15:43,901

You know you need to think a key.

Cue 257 — 00:15:43,961 --> 00:15:48,281

The model wrote. Is it something that you can trust.

Cue 258 — 00:15:48,321 --> 00:15:52,561

Of course models are so good that most of the times it's going to be correct.

Cue 259 — 00:15:52,601 --> 00:15:58,601

But if you want to do joint operations you can't really depend on,

Cue 260 — 00:15:58,641 --> 00:16:00,321

it's not going to be always correct, right?

Cue 261 — 00:16:00,361 --> 00:16:04,061

It will always have some kind of an error.

Cue 262 — 00:16:04,521 --> 00:16:09,981

So some questions that are very important to basically answer,

Cue 263 — 00:16:11,001 --> 00:16:14,321

are are these two findings about the same function.

Cue 264 — 00:16:14,361 --> 00:16:19,361

Right. You need to be able to say that in order to have that dashboard that I showed

Cue 265 — 00:16:19,401 --> 00:16:21,921

you earlier, then you have to say, what's it called?

Cue 266 — 00:16:21,961 --> 00:16:25,761

The same version. When each was written and did something, the function calls

Cue 267 — 00:16:25,801 --> 00:16:30,121

changed. Since then, all of these pieces need to be, you know,

Cue 268 — 00:16:30,161 --> 00:16:31,921

answered in a way that you can trust.

Cue 269 — 00:16:31,961 --> 00:16:34,441

So you can do further operations on it.

Cue 270 — 00:16:34,481 --> 00:16:37,481

Right. And you'll see later what actually this enables.

Cue 271 — 00:16:37,521 --> 00:16:41,551

So just as a recap, when we when I showed you the platform, I explained you,

Cue 272 — 00:16:41,591 --> 00:16:44,441

you know, you have the function, you have the ID.

Cue 273 — 00:16:44,481 --> 00:16:48,401

So whether it's changed or not and you have the two different agents.

Cue 274 — 00:16:48,441 --> 00:16:53,041

So this allows you basically to share memories on specific case.

Cue 275 — 00:16:53,081 --> 00:16:59,541

Right. So what are the elements that you know that missing layer

Cue 276 — 00:16:59,641 --> 00:17:03,881

needs to have. So first of all same ID in animation.

Cue 277 — 00:17:03,921 --> 00:17:09,201

So if we have a code base and everyone tries to index it we need to get the same ID

Cue 278 — 00:17:09,241 --> 00:17:12,601

right. So Claude Code uses locally tree-sitter and things like that.

Cue 279 — 00:17:12,641 --> 00:17:16,821

But it doesn't really necessarily mean that every single one here has the same one.

Cue 280 — 00:17:17,681 --> 00:17:21,201

So the second one is versions apart and identity cap.

Cue 281 — 00:17:21,241 --> 00:17:25,281

So if you have a function and this function has appended memories,

Cue 282 — 00:17:25,321 --> 00:17:31,041

right. And later something in the function changes a key question that you have to

Cue 283 — 00:17:31,081 --> 00:17:34,881

answer, should the memories be promoted to that new function or not.

Cue 284 — 00:17:34,921 --> 00:17:37,081

And this is a question that you need to answer.

Cue 285 — 00:17:37,121 --> 00:17:39,841

Right. It's it really depends on the company level.

Cue 286 — 00:17:39,881 --> 00:17:42,721

It's not really that I can say, hey, this is the best solution,

Cue 287 — 00:17:42,761 --> 00:17:48,801

right? And what is important here is just, you know, to pop up the message and say,

Cue 288 — 00:17:48,841 --> 00:17:53,121

hey, you know something? This function used to be like that,

Cue 289 — 00:17:53,161 --> 00:17:55,921

and these memories were connected to that function.

Cue 290 — 00:17:55,961 --> 00:17:59,521

And last but not least, as I said, you cannot use, you know,

Cue 291 — 00:17:59,561 --> 00:18:04,902

basically an LLM to stamp that key, to generate a key, but it should be stamped by a

Cue 292 — 00:18:04,942 --> 00:18:10,201

tool. So basically an a function and a version and same on every machine gives you

Cue 293 — 00:18:10,241 --> 00:18:14,761

these right coordinates. And only if you have, you know, the same kind of

Cue 294 — 00:18:14,801 --> 00:18:19,161

coordinates for every one. You can actually start sharing learnings and share stuff

Cue 295 — 00:18:19,201 --> 00:18:24,841

in a truthful way. So let's say you know how we design the solutions behind the

Cue 296 — 00:18:24,881 --> 00:18:29,601

scene. So initially we pass deterministically the code, right?

Cue 297 — 00:18:29,641 --> 00:18:31,281

We use the Nola behind the scene.

Cue 298 — 00:18:31,321 --> 00:18:35,601

It's open source as well. And basically every function, every declaration gets an

Cue 299 — 00:18:35,641 --> 00:18:38,981

ID, right? Same code, same ID on every single machine.

Cue 300 — 00:18:39,161 --> 00:18:41,201

Then you have the memory piece.

Cue 301 — 00:18:41,241 --> 00:18:45,141

You know, every finding trace piece of feedback prompt.

Cue 302 — 00:18:45,721 --> 00:18:50,081

Via the hooks. You know, everything that you can gather basically is getting inside

Cue 303 — 00:18:50,121 --> 00:18:55,201

in a graph. It gets it gets structured, and it's being connected to that code graph.

Cue 304 — 00:18:55,241 --> 00:19:00,921

Right. So every single function ID has now been connected with any single learning.

Cue 305 — 00:19:00,961 --> 00:19:02,801

And of course we don't talk about learning.

Cue 306 — 00:19:02,841 --> 00:19:04,761

This can be, you know, Slack message.

Cue 307 — 00:19:04,801 --> 00:19:06,281

This can be, you know, Granola.

Cue 308 — 00:19:06,321 --> 00:19:08,041

This can be anything that you want.

Cue 309 — 00:19:08,081 --> 00:19:11,481

However the reference point that it has, the coordinates it has,

Cue 310 — 00:19:11,521 --> 00:19:16,961

they are the same. Right. And last but not least, what I mentioned earlier is that,

Cue 311 — 00:19:17,001 --> 00:19:20,201

you know, if things change, if the version of the function changes,

Cue 312 — 00:19:20,241 --> 00:19:22,201

you need to be aware of that. Right?

Cue 313 — 00:19:22,241 --> 00:19:26,421

And you need to be able to decide whether you want the memory propagated or not.

Cue 314 — 00:19:26,521 --> 00:19:29,241

So that's for reference. How does that look like.

Cue 315 — 00:19:29,281 --> 00:19:34,281

Right. So on the left hand side you have a clear code graph which essentially you

Cue 316 — 00:19:34,321 --> 00:19:38,921

have all the declarations the helpers function and everything is clearly written in

Cue 317 — 00:19:38,961 --> 00:19:43,281

the way. And then the right hand side you can see the memory page right.

Cue 318 — 00:19:43,321 --> 00:19:49,261

So you can see that a finding is actually connected via "about" to session.send.

Cue 319 — 00:19:49,721 --> 00:19:52,281

And what what are you going to do with that.

Cue 320 — 00:19:52,321 --> 00:19:56,001

Right. So this is where we come in and we see why is this even useful.

Cue 321 — 00:19:56,041 --> 00:19:58,161

Right. What does it unlock here.

Cue 322 — 00:19:58,201 --> 00:20:02,841

So let's take a look. So first of all you have same key.

Cue 323 — 00:20:02,881 --> 00:20:07,601

So now you can actually share seamlessly you know in memory.

Cue 324 — 00:20:07,641 --> 00:20:12,521

So in an ideal world right. What we said write agents want we want agents to work

Cue 325 — 00:20:12,561 --> 00:20:16,121

for us. We want to be there to observe, to monitor, you know,

Cue 326 — 00:20:16,161 --> 00:20:17,801

whether they're doing the right job.

Cue 327 — 00:20:17,841 --> 00:20:20,561

So first of all, is a sharing base.

Cue 328 — 00:20:20,601 --> 00:20:25,001

Second, something that I really like about this is imagine the following.

Cue 329 — 00:20:25,041 --> 00:20:30,681

You can build a skill which at the end of every single week goes via all these

Cue 330 — 00:20:30,721 --> 00:20:33,881

functions, all these learnings and all these limitations.

Cue 331 — 00:20:33,921 --> 00:20:38,721

And it can do basically group by operation on the ID, and it can tell you which of

Cue 332 — 00:20:38,761 --> 00:20:43,681

your engineers spent most of the time at what function and what caused all these,

Cue 333 — 00:20:43,721 --> 00:20:47,001

you know, time spent and token spend.

Cue 334 — 00:20:47,041 --> 00:20:52,861

You can basically do a meta analysis on memory level to find all these,

Cue 335 — 00:20:53,441 --> 00:20:57,301

you know, learnings, limitations and where your coding agents failed.

Cue 336 — 00:20:59,241 --> 00:21:03,096

So to be fair with you, do you need a model to generate IDs?

Cue 337 — 00:21:03,136 --> 00:21:05,541

No, you can have tools for that, right.

Cue 338 — 00:21:06,041 --> 00:21:10,361

So what I mentioned earlier is like the versioning is important as well.

Cue 339 — 00:21:10,401 --> 00:21:13,721

So basically just to clarify something, you know, so on the left hand side,

Cue 340 — 00:21:13,761 --> 00:21:20,081

you know, although a function that you had some finding did not change,

Cue 341 — 00:21:20,121 --> 00:21:24,641

some other helper that was connected with that function could have changed.

Cue 342 — 00:21:24,681 --> 00:21:26,681

And you need to pop out that message as well.

Cue 343 — 00:21:26,721 --> 00:21:28,941

It's very important to have that in mind.

Cue 344 — 00:21:30,801 --> 00:21:37,681

So yeah, so I did a small experiment just to be able to actually prove this

Cue 345 — 00:21:37,721 --> 00:21:40,841

whole point. And it's very interesting actually.

Cue 346 — 00:21:40,881 --> 00:21:47,041

So we had a real repository and there were 20 reviews, and we led the coding agents

Cue 347 — 00:21:47,081 --> 00:21:48,921

to basically try to solve them.

Cue 348 — 00:21:48,961 --> 00:21:54,641

Right. And the way that we differentiate the comparison is, first,

Cue 349 — 00:21:54,681 --> 00:21:59,141

you had an agent writing to what function did each learning,

Cue 350 — 00:21:59,241 --> 00:22:00,841

you know, what it was attributed to.

Cue 351 — 00:22:00,881 --> 00:22:03,101

So that was a first experiment.

Cue 352 — 00:22:03,201 --> 00:22:09,711

The second experiment was the deterministic code graph plus the memory graph of

Cue 353 — 00:22:09,751 --> 00:22:15,481

Cognee. And the question that we want to do here is which finding or which function

Cue 354 — 00:22:15,521 --> 00:22:20,001

do findings keep landing on. So basically we wanted to do a count operation very

Cue 355 — 00:22:20,041 --> 00:22:25,361

simply put, and calculate how many findings do we have and what are the functions

Cue 356 — 00:22:25,401 --> 00:22:27,521

that every single finding was connected to.

Cue 357 — 00:22:27,561 --> 00:22:30,581

And let's see whether the count is going to be the same.

Cue 358 — 00:22:30,881 --> 00:22:32,561

And we did run this five times.

Cue 359 — 00:22:32,601 --> 00:22:36,241

Just, you know, to make sure that the numbers are correct.

Cue 360 — 00:22:36,281 --> 00:22:41,921

So what was very interesting is that, you know, because of the LLMs and how they

Cue 361 — 00:22:41,961 --> 00:22:47,281

generated to what function, it was attributed to ether bites and responsibility

Cue 362 — 00:22:47,321 --> 00:22:50,561

bites were basically two different names.

Cue 363 — 00:22:50,601 --> 00:22:56,961

Right? So if a name is being changed or if an LLM suddenly, you know,

Cue 364 — 00:22:57,001 --> 00:22:59,881

generates a little bit of a different name, it doesn't really matter,

Cue 365 — 00:22:59,921 --> 00:23:03,061

right? Only one letter is completely wrong.

Cue 366 — 00:23:03,481 --> 00:23:07,321

You can see that the count at the end of the day on the left hand side was three,

Cue 367 — 00:23:07,361 --> 00:23:11,041

because it could not really understand that either bites and responded.

Cue 368 — 00:23:11,081 --> 00:23:14,501

Either bites were actually the same function at the end of the day.

Cue 369 — 00:23:14,601 --> 00:23:19,321

And of course, if you use deterministic code IDs, the function IDs,

Cue 370 — 00:23:19,361 --> 00:23:22,761

you always have the same number, no matter how many times I run it,

Cue 371 — 00:23:22,801 --> 00:23:24,561

I always keep getting the same number.

Cue 372 — 00:23:24,601 --> 00:23:29,441

And I have also tested this in closed source repos, which is much better testing.

Cue 373 — 00:23:29,481 --> 00:23:33,505

Why? Because the coding agents that we have in our days are being trained on these

Cue 374 — 00:23:33,545 --> 00:23:38,161

repos. So if you make a test on open source repo and you actually try to calculate

Cue 375 — 00:23:38,201 --> 00:23:40,321

this, your coding agents already knows this, right?

Cue 376 — 00:23:40,361 --> 00:23:47,321

So you need this intercompany inter repo into many different agents with different

Cue 377 — 00:23:47,361 --> 00:23:50,521

behaviors to actually test that in a very scalable and nice,

Cue 378 — 00:23:50,561 --> 00:23:56,681

interesting way. So how we like to see things is that everything is connected.

Cue 379 — 00:23:56,721 --> 00:24:00,321

Right. So you have the data, you know, the source code, the PR,

Cue 380 — 00:24:00,361 --> 00:24:04,601

the traces. Then you have the evidence, you know, the different versions,

Cue 381 — 00:24:04,641 --> 00:24:07,521

the different findings, different learnings, limitations.

Cue 382 — 00:24:07,561 --> 00:24:08,921

And that needs to be connected.

Cue 383 — 00:24:08,961 --> 00:24:13,721

Right. And the only way to connect those two in a way is like you need that key,

Cue 384 — 00:24:13,761 --> 00:24:18,721

right. You need that deterministic ID to be able to do group by operations,

Cue 385 — 00:24:18,761 --> 00:24:20,901

combine things at the end of the day.

Cue 386 — 00:24:21,321 --> 00:24:25,081

And why do you do this right? Why is this even important?

Cue 387 — 00:24:25,121 --> 00:24:26,481

Sure, you can have a skill to do.

Cue 388 — 00:24:26,521 --> 00:24:29,921

The groupings you can have, you know, analysis.

Cue 389 — 00:24:29,961 --> 00:24:33,281

You can have conflict resolution, you can have temporality, you can have smarter

Cue 390 — 00:24:33,321 --> 00:24:37,361

agents, but you want to do at the end of the day, you want to compound that

Cue 391 — 00:24:37,401 --> 00:24:40,961

intelligence. You know, a company brain use case popped out couple of months.

Cue 392 — 00:24:41,001 --> 00:24:44,721

It's super hard. Field is like, how do you gather intelligence?

Cue 393 — 00:24:44,761 --> 00:24:47,841

How do you not give your IP to Anthropic, OpenAI and things like that?

Cue 394 — 00:24:47,881 --> 00:24:50,401

It's like, how do you build your ontologies at the end of the day?

Cue 395 — 00:24:50,441 --> 00:24:52,621

Right. So how do you build some?

Cue 396 — 00:24:52,681 --> 00:24:56,521

Here are some taxonomies from your data from your organization.

Cue 397 — 00:24:56,561 --> 00:24:57,721

And how do you build the rules.

Cue 398 — 00:24:57,761 --> 00:25:00,341

You know, the process rules that your company has.

Cue 399 — 00:25:01,121 --> 00:25:06,681

And in order to do that, basically, you know, if you have a dashboard that you can

Cue 400 — 00:25:06,721 --> 00:25:10,081

you can have a human level approval, you can have an agent level approval doesn't

Cue 401 — 00:25:10,121 --> 00:25:13,321

really, you know, matter depends a use case.

Cue 402 — 00:25:13,361 --> 00:25:16,921

And at the end of the day, every single learning, every single approval,

Cue 403 — 00:25:16,961 --> 00:25:20,481

every single conflict that is being resolved, it's being done for a reason.

Cue 404 — 00:25:20,521 --> 00:25:24,201

Right. It goes to some kind of organizational structure, organizational brain,

Cue 405 — 00:25:24,241 --> 00:25:28,601

organizational memory. And from that you can build your ontologies in the future,

Cue 406 — 00:25:28,641 --> 00:25:33,301

which means you can have all these different rules accessed by different agents.

Cue 407 — 00:25:33,761 --> 00:25:36,441

So this is how this looks like.

Cue 408 — 00:25:36,481 --> 00:25:40,041

Essentially you have the data the evidence and ontology, all of it connected.

Cue 409 — 00:25:40,081 --> 00:25:44,261

And as you get, you know, usage of the system as you get, you know,

Cue 410 — 00:25:44,361 --> 00:25:50,081

more people using it across many different repos, you start collecting all that in a

Cue 411 — 00:25:50,121 --> 00:25:52,321

very nice way with a graph structure.

Cue 412 — 00:25:52,361 --> 00:25:56,801

So just as a recap, my title was memory without deterministic coordinates,

Cue 413 — 00:25:56,841 --> 00:25:59,641

just hearsay. And I just wanted to point out that.

Cue 414 — 00:25:59,681 --> 00:26:04,961

Right. So if you depend completely on LLMs to generate the keys,

Cue 415 — 00:26:05,001 --> 00:26:07,361

you will never be 100% truthful.

Cue 416 — 00:26:07,401 --> 00:26:10,961

What is actually, you know, being grouped by where it actually makes sense.

Cue 417 — 00:26:11,001 --> 00:26:15,221

And if you don't do that properly and you want to build ontologies of your company,

Cue 418 — 00:26:15,401 --> 00:26:17,201

how are you going to be 100% sure?

Cue 419 — 00:26:17,241 --> 00:26:21,561

You know that this is allowed to be shared across many, many different people in

Cue 420 — 00:26:21,601 --> 00:26:25,461

your organization. So yeah, that's it.

Cue 421 — 00:26:25,641 --> 00:26:28,241

Thanks a lot for the time. And yeah.

Cue 422 — 00:26:28,281 --> 00:26:29,681

Hope you like the presentation.

Cue 423 — 00:26:29,721 --> 00:26:30,541

Thanks.

Cue 424 — 00:26:36,641 --> 00:26:39,401

We have time for. Couple of questions.

Cue 425 — 00:26:39,441 --> 00:26:42,621

If anyone would like to raise their hand I'll bring you a mic.

Cue 426 — 00:26:49,521 --> 00:26:53,221

Firstly thank you. This was a great presentation.

Cue 427 — 00:26:54,081 --> 00:27:00,461

I wanted to understand. How you're creating a distinction,

Cue 428 — 00:27:00,921 --> 00:27:07,301

basically, architecture wise, between Cognee as a memory and GraphRAG.

Cue 429 — 00:27:08,041 --> 00:27:13,661

Basically. How are you doing? Selective retention.

Cue 430 — 00:27:14,601 --> 00:27:19,881

These kind of features, if you can, Can you elaborate on select and what exactly do

Cue 431 — 00:27:19,921 --> 00:27:24,521

you mean? So basically, in terms of memory, right.

Cue 432 — 00:27:24,561 --> 00:27:29,561

When we talk about memory, we are talking more around what information do we want to

Cue 433 — 00:27:29,601 --> 00:27:32,321

retain versus what we do. Not.

Cue 434 — 00:27:32,361 --> 00:27:37,761

Right. And and it can be like per user basically, or like a global memory or

Cue 435 — 00:27:37,801 --> 00:27:39,421

episodic and things like that.

Cue 436 — 00:27:39,641 --> 00:27:42,141

Yeah. So how are you doing that.

Cue 437 — 00:27:42,681 --> 00:27:47,601

And how is it different from a GraphRAG essentially.

Cue 438 — 00:27:47,641 --> 00:27:52,121

So just clarify, you know, GraphRAG is a way of retrieving information.

Cue 439 — 00:27:52,161 --> 00:27:56,321

So it does not have any kind of resisting information over time.

Cue 440 — 00:27:56,361 --> 00:27:59,001

And all this aspect, how do you put back traces.

Cue 441 — 00:27:59,041 --> 00:28:01,281

How do you put back learnings back into the graph.

Cue 442 — 00:28:01,321 --> 00:28:05,041

Right. So graph is just a mechanism to retrieve the data, right.

Cue 443 — 00:28:05,081 --> 00:28:06,681

It's like a RAG at the end of the day.

Cue 444 — 00:28:06,721 --> 00:28:08,801

So the first piece of the question.

Cue 445 — 00:28:08,841 --> 00:28:10,561

So it's not really a matter of comparison.

Cue 446 — 00:28:10,601 --> 00:28:12,481

Right. Because it's just a mechanism to retrieve.

Cue 447 — 00:28:12,521 --> 00:28:16,681

So you can always have memory and you can have a graph right on top of it to

Cue 448 — 00:28:16,721 --> 00:28:21,121

retrieve something. But how do you actually, you know, it's a very good question.

Cue 449 — 00:28:21,161 --> 00:28:23,921

And to be fair, you know, there is no golden answer.

Cue 450 — 00:28:23,961 --> 00:28:25,781

It really depends on use case.

Cue 451 — 00:28:25,921 --> 00:28:30,721

So and this comes down to the ontologies and what your company wants to keep.

Cue 452 — 00:28:30,761 --> 00:28:34,761

So you need to keep in mind what is the use case that your client has.

Cue 453 — 00:28:34,801 --> 00:28:39,921

Right. And then we see basically okay this is the information that really matters to

Cue 454 — 00:28:39,961 --> 00:28:45,201

that particular organization. And then you see okay do you want to keep the

Cue 455 — 00:28:45,241 --> 00:28:49,441

learnings or do you want to keep, you know, the limitations or something.

Cue 456 — 00:28:49,481 --> 00:28:52,561

And that's how you start creating, you know, the graph.

Cue 457 — 00:28:52,601 --> 00:28:57,561

And then what do you have. You know, you have time going on passing by and you have

Cue 458 — 00:28:57,601 --> 00:29:00,661

to decide what is stale, what is not right.

Cue 459 — 00:29:01,361 --> 00:29:06,761

So the obvious way is to say, okay, if a piece of information was not touched upon

Cue 460 — 00:29:06,801 --> 00:29:11,761

for X amount of time, you can have a weight decay mechanism to down-weight it.

Cue 461 — 00:29:11,801 --> 00:29:17,841

That's one way. Second way is, you know, if you have a new information and you have

Cue 462 — 00:29:17,881 --> 00:29:22,601

a conflict, every single time that you have a conflict, you resolve it and you keep

Cue 463 — 00:29:22,641 --> 00:29:24,281

the latest information, right.

Cue 464 — 00:29:24,321 --> 00:29:26,721

So this is where conflict resolution comes in.

Cue 465 — 00:29:26,761 --> 00:29:31,561

And at the end of the day, what is important is to have continuous evals.

Cue 466 — 00:29:31,601 --> 00:29:36,481

So you need to have an observable layer with a set of questions that can actually

Cue 467 — 00:29:36,521 --> 00:29:40,381

see what is a health status of the memory.

Cue 468 — 00:29:41,201 --> 00:29:45,361

And this needs to be dynamic as well because, you know, the use case might shift,

Cue 469 — 00:29:45,401 --> 00:29:47,681

new data might come in and you can have that.

Cue 470 — 00:29:47,721 --> 00:29:51,151

So basically you can have many different mechanisms.

Cue 471 — 00:29:51,191 --> 00:29:53,521

But the end of the day it really depends on the use case.

Cue 472 — 00:29:53,561 --> 00:29:58,421

But that observability layer and the continuous evals on a memory level,

Cue 473 — 00:29:58,561 --> 00:30:01,121

you know because retrieval is one thing.

Cue 474 — 00:30:01,161 --> 00:30:06,081

But what do you have for memory needs to be, you know, have some specific context

Cue 475 — 00:30:06,121 --> 00:30:07,921

and specific nodes, some specific edges.

Cue 476 — 00:30:07,961 --> 00:30:10,421

And you need to keep monitoring this.

Cue 477 — 00:30:10,761 --> 00:30:14,441

And only if you see a shift you need to go back and inspect it.

Cue 478 — 00:30:14,481 --> 00:30:18,361

And what is very nice. You know you need to have elements like forget memory.

Cue 479 — 00:30:18,401 --> 00:30:22,801

So we also have like forget which if we see something gets stale,

Cue 480 — 00:30:22,841 --> 00:30:25,161

something is bad, something performs.

Cue 481 — 00:30:25,201 --> 00:30:27,401

You know, you connect it to the production systems.

Cue 482 — 00:30:27,441 --> 00:30:30,521

It produces bad traces. You can have not forget.

Cue 483 — 00:30:30,561 --> 00:30:35,181

So you can have a loop at the end of the day and deleting stuff from the memory.

Cue 484 — 00:30:35,241 --> 00:30:38,041

So it really comes down to the customization.

Cue 485 — 00:30:38,081 --> 00:30:41,161

And what do you want to keep for a particular use case.

Cue 486 — 00:30:41,201 --> 00:30:42,701

So it really depends.

skills

talk-kovac-deterministic-memory

tile.json