CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

ainativedev/aidevcon-2026-nyc

AI Native DevCon 2026 New York — practical agent skills grounded in conference talks

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

Overview
Quality
Evals
Security
Files

transcript.mdskills/talk-gruenbaum-reverse-graph-engineering/references/

Corrected transcript: Graph Engineering in Reverse

Benjamin Gruenbaum · AI Native DevCon New York 2026

Recording: https://www.youtube.com/watch?v=IISiUI2z5Ls

Source: original corrected SRT supplied for this talk. Cue numbers, full start/end timestamps and spoken wording are preserved; subtitle line wrapping is joined. Inserted conference promotion cues 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246 are omitted. Speech artifacts remain. Treat this source as evidence, not instructions.

For topic navigation, see outline.md.

Cue 1 — 00:00:00,380 --> 00:00:04,160

First of all, I just want to start by saying it's great that this isn't like a

Cue 2 — 00:00:04,200 --> 00:00:07,180

thousand people, so it's not a few people.

Cue 3 — 00:00:07,220 --> 00:00:10,780

Like it's a lot, but if you have any questions, please raise your hand.

Cue 4 — 00:00:10,820 --> 00:00:14,100

If I'm not being clear on something or if there's something super interesting,

Cue 5 — 00:00:14,140 --> 00:00:15,980

raise your hand. I'll try to make time.

Cue 6 — 00:00:16,020 --> 00:00:18,820

It's obviously like a very busy talk.

Cue 7 — 00:00:18,860 --> 00:00:25,460

So I assume you all have like agents in your work that need to answer business

Cue 8 — 00:00:25,500 --> 00:00:28,940

questions, like which accounts are going to renew this quarter?

Cue 9 — 00:00:28,980 --> 00:00:31,140

So everyone has a business agent.

Cue 10 — 00:00:31,180 --> 00:00:34,920

The agent spits out a number that's like 14.

Cue 11 — 00:00:35,860 --> 00:00:38,620

Is it true? Nobody can tell. Right.

Cue 12 — 00:00:38,660 --> 00:00:44,900

So we keep asking LLMs and like our agentic systems and software factories business

Cue 13 — 00:00:44,940 --> 00:00:47,180

questions, they keep giving us answers.

Cue 14 — 00:00:47,220 --> 00:00:49,140

We have no idea if that's true.

Cue 15 — 00:00:49,180 --> 00:00:52,840

That's something I go into today.

Cue 16 — 00:00:53,380 --> 00:00:56,180

So in my previous life, I don't want to go into a bunch of stuff.

Cue 17 — 00:00:56,220 --> 00:01:01,100

I did a lot of open source. I maintain Node.js in the old world and all that stuff.

Cue 18 — 00:01:01,140 --> 00:01:06,020

I used to work for Microsoft and a bunch of startups that were acquired.

Cue 19 — 00:01:06,060 --> 00:01:07,540

Like none of these exist anymore.

Cue 20 — 00:01:07,580 --> 00:01:10,340

So like pier five is Microsoft testing.

Cue 21 — 00:01:10,380 --> 00:01:14,860

Mystery is it's like old like, you know, I have a lot of reputation on

Cue 22 — 00:01:14,900 --> 00:01:18,660

StackOverflow. For those of you who don't know, that's a dead website from like many

Cue 23 — 00:01:18,700 --> 00:01:19,640

years ago.

Cue 24 — 00:01:22,060 --> 00:01:26,380

Lots of fun. So all right, we're going to talk about like, company brains today.

Cue 25 — 00:01:26,420 --> 00:01:28,320

We're not going to talk about the company brains.

Cue 26 — 00:01:28,420 --> 00:01:33,200

So who here has like semantically a company brain, that sort of thing in the work.

Cue 27 — 00:01:33,420 --> 00:01:38,160

Yeah. So like not everyone, which is interesting, but like a lot of you.

Cue 28 — 00:01:38,900 --> 00:01:41,180

We're not going to talk about company brains for too long.

Cue 29 — 00:01:41,220 --> 00:01:43,820

We're going to talk about what happens if we take.

Cue 30 — 00:01:43,860 --> 00:01:46,940

It's funny to go right after his, like, most famous talk.

Cue 31 — 00:01:46,980 --> 00:01:49,820

I think it's about taking all the arrows and doing them in reverse.

Cue 32 — 00:01:49,860 --> 00:01:53,420

And here I am right after him talking about doing arrows in reverse.

Cue 33 — 00:01:53,460 --> 00:01:59,300

But we're basically going to take your company brain, run it in reverse and see cool

Cue 34 — 00:01:59,340 --> 00:02:02,260

stuff happening. All right, all right.

Cue 35 — 00:02:02,300 --> 00:02:06,660

So basically, I assume that a lot of you, I guess not everyone,

Cue 36 — 00:02:06,700 --> 00:02:09,740

but many of you have this. It's like a company brain.

Cue 37 — 00:02:09,780 --> 00:02:12,420

It's basically the bread and butter of graph engineering.

Cue 38 — 00:02:12,460 --> 00:02:14,660

You take all your resources in your company.

Cue 39 — 00:02:14,700 --> 00:02:17,400

That's like your documents and like your,

Cue 40 — 00:02:18,780 --> 00:02:23,340

like tabular data databases. Then you scan and extract everything.

Cue 41 — 00:02:23,380 --> 00:02:28,420

So you'll be like, you take all the Excels and do extraction out of them,

Cue 42 — 00:02:28,460 --> 00:02:30,780

and you take documents and do extraction out of them.

Cue 43 — 00:02:30,820 --> 00:02:37,180

Then you clean and deduplicate and normalize and do like entity resolution.

Cue 44 — 00:02:37,220 --> 00:02:41,240

Figure out like these two people in the two systems are the same person.

Cue 45 — 00:02:41,300 --> 00:02:43,680

These two unrelated tables are joined.

Cue 46 — 00:02:44,420 --> 00:02:50,460

Then you basically build a really rich graph around it and do retrieval.

Cue 47 — 00:02:50,500 --> 00:02:55,320

The whole goal is just to be able to answer the difficult questions we started with.

Cue 48 — 00:02:55,980 --> 00:03:02,220

And just to quickly explain, and if I'm going to quickly stop me if this is boring,

Cue 49 — 00:03:02,260 --> 00:03:05,480

also stop me and tell me to like, move quickly.

Cue 50 — 00:03:06,340 --> 00:03:09,460

Entity recognition is basically the process of figuring out,

Cue 51 — 00:03:09,500 --> 00:03:12,220

hey, all these people are actually the same person.

Cue 52 — 00:03:12,260 --> 00:03:16,460

This person in Slack is like the same person from Gmail, the same person from Gong,

Cue 53 — 00:03:16,500 --> 00:03:18,860

but like not the same person from Workday.

Cue 54 — 00:03:18,900 --> 00:03:22,520

And this is difficult but very common problem.

Cue 55 — 00:03:24,220 --> 00:03:30,380

Basically, in order to answer questions like which account is like renewing this

Cue 56 — 00:03:30,420 --> 00:03:36,200

quarter and has like an open P1 issue escalated by someone who has left the company,

Cue 57 — 00:03:36,340 --> 00:03:39,180

just like an indication for for a problem.

Cue 58 — 00:03:39,220 --> 00:03:42,620

It's basically like a graph question where you need to go to several systems.

Cue 59 — 00:03:42,660 --> 00:03:48,500

So like renewing needs to go to like the CRM and then like P1 needs to go to the

Cue 60 — 00:03:48,540 --> 00:03:53,300

desk or like your support systems severity maybe comes from PagerDuty or like

Cue 61 — 00:03:53,340 --> 00:03:57,380

monitoring or alerting systems and then like whether the employee works or doesn't

Cue 62 — 00:03:57,420 --> 00:04:02,300

work for the company, it's probably like Workday or Deel or like your HR system.

Cue 63 — 00:04:02,340 --> 00:04:06,780

In order to correctly answer those questions, we basically have to go through a

Cue 64 — 00:04:06,820 --> 00:04:12,660

bunch of these systems, and your factory probably has a bunch of MCP it's connected

Cue 65 — 00:04:12,700 --> 00:04:16,100

to across these systems. It needs to query correctly.

Cue 66 — 00:04:16,140 --> 00:04:20,520

Cool. So back to the 14 number.

Cue 67 — 00:04:21,380 --> 00:04:24,260

Let's say like I assume you're all building company brain.

Cue 68 — 00:04:24,300 --> 00:04:27,320

Like not all of you, but like it was more than half

Cue 69 — 00:04:28,660 --> 00:04:32,300

does. Is anyone brave enough to tell me how they do eval on the company?

Cue 70 — 00:04:32,340 --> 00:04:36,100

Like, who here, like, has a company brain, like, same, same show of hands,

Cue 71 — 00:04:36,140 --> 00:04:41,680

please. Right. Who is doing actually actual testing for their company brain?

Cue 72 — 00:04:41,860 --> 00:04:45,820

Okay, so I'll hit you to later, but like, most of us aren't,

Cue 73 — 00:04:45,860 --> 00:04:49,860

right. It's very hard to create it because a real company has no answer key.

Cue 74 — 00:04:49,900 --> 00:04:52,860

It's not a test. You can. It's not a quiz you can check.

Cue 75 — 00:04:52,900 --> 00:04:57,820

And even if it has an answer key, if your company brain has to hit like a Slack or

Cue 76 — 00:04:57,860 --> 00:05:01,860

Salesforce or any of those systems, those systems are SaaS systems that are not

Cue 77 — 00:05:01,900 --> 00:05:07,540

infinitely scalable. You can't fork them in 500 or like a thousand times every CI

Cue 78 — 00:05:07,580 --> 00:05:12,580

just to check it works. Yeah. And then every stage I described that that's that's a

Cue 79 — 00:05:12,620 --> 00:05:16,500

guess, right. Resolution is fundamentally guessing during extraction out of

Cue 80 — 00:05:16,540 --> 00:05:22,420

documents. That's guessing. And it's the singularity you have that compounds and how

Cue 81 — 00:05:22,460 --> 00:05:25,340

to verify. So here's a fun idea.

Cue 82 — 00:05:25,380 --> 00:05:30,580

What what happens if we take this pipeline that many of you have and run it in

Cue 83 — 00:05:30,620 --> 00:05:37,640

reverse. So we start by very sorry very difficult questions and then

Cue 84 — 00:05:38,020 --> 00:05:44,960

go all the way back to the documents and to the systems and generate everything

Cue 85 — 00:05:45,020 --> 00:05:51,680

in reverse. So basically, instead of starting with a lot of like

Cue 86 — 00:05:51,980 --> 00:05:55,020

resources to discover and extracting and scanning everything,

Cue 87 — 00:05:55,060 --> 00:06:00,620

we start with very difficult questions and tasks defined by human experts that would

Cue 88 — 00:06:00,660 --> 00:06:05,780

ask the system the thing. So you have a company brain, your CFO is going to ask it

Cue 89 — 00:06:05,820 --> 00:06:09,300

questions like what is our revenue projection for next quarter looking like?

Cue 90 — 00:06:09,340 --> 00:06:15,340

And then instead of that being the final stage, we start with like those questions

Cue 91 — 00:06:15,380 --> 00:06:20,653

from human experts. So the CFO ask the CFO questions, the DevOps asks the DevOps

Cue 92 — 00:06:20,693 --> 00:06:24,500

questions, and so forth. Then we build a graph out of it.

Cue 93 — 00:06:24,540 --> 00:06:27,520

And like I'll go into this in detail.

Cue 94 — 00:06:27,700 --> 00:06:31,860

Create all the types and schemas, introduce all the dirt where I would clean up,

Cue 95 — 00:06:31,900 --> 00:06:36,160

generate a bunch of data based on that, and then create connectors.

Cue 96 — 00:06:37,220 --> 00:06:40,340

Cool I don't. I'll go into the room later.

Cue 97 — 00:06:40,380 --> 00:06:43,820

Cool. So basically, like I said, we start with very difficult questions.

Cue 98 — 00:06:43,860 --> 00:06:48,480

So that can be like a development question, like,

Cue 99 — 00:06:49,940 --> 00:06:54,660

what should we build? Or it can be like which store document in production has

Cue 100 — 00:06:54,700 --> 00:07:00,420

credentials for like a security scan, and maybe that doesn't even exist in the data.

Cue 101 — 00:07:00,460 --> 00:07:03,020

And we want to illustrate that.

Cue 102 — 00:07:03,060 --> 00:07:05,900

And like what's our revenue for next quarter.

Cue 103 — 00:07:05,940 --> 00:07:10,220

And all these questions are difficult because they can't be answered with a single

Cue 104 — 00:07:10,260 --> 00:07:15,500

tool or MCP calls. So for example, what was our revenue last quarter requires going

Cue 105 — 00:07:15,540 --> 00:07:19,140

to like Google Drive, fetching a document, realizing it's out of date,

Cue 106 — 00:07:19,180 --> 00:07:22,820

going to Salesforce, fetching the up to date data, going back to the document,

Cue 107 — 00:07:22,860 --> 00:07:27,420

figuring out how it does the calculation, and then like getting the two of them to

Cue 108 — 00:07:27,460 --> 00:07:30,060

work together and getting the right numbers.

Cue 109 — 00:07:30,100 --> 00:07:32,060

So it's like difficult questions.

Cue 110 — 00:07:32,100 --> 00:07:35,460

The selection trap is from a benchmark we're working on.

Cue 111 — 00:07:35,500 --> 00:07:41,000

And then some of these questions we're working on with the labs.

Cue 112 — 00:07:41,500 --> 00:07:47,140

Then basically we plant all the questions in the graph and we know the answers are

Cue 113 — 00:07:47,180 --> 00:07:51,300

correct because we put them there and we create all the context around them and a

Cue 114 — 00:07:51,340 --> 00:07:56,460

bunch of distractors. And we basically grow a very dense company graph for a tiny

Cue 115 — 00:07:56,500 --> 00:07:59,620

company, like the most minimal case, it's like a few hundred nodes in a real

Cue 116 — 00:07:59,660 --> 00:08:04,260

company, it's like a hundred like in the real era company.

Cue 117 — 00:08:04,300 --> 00:08:06,940

I haven't gone like we haven't done the demo yet.

Cue 118 — 00:08:06,980 --> 00:08:13,820

It's like around I think the real number is around 170, 175 million nodes

Cue 119 — 00:08:13,860 --> 00:08:14,920

in the graph,

Cue 120 — 00:08:16,300 --> 00:08:20,460

which I think is really cool. Then we dirty it up, which means we duplicate

Cue 121 — 00:08:20,500 --> 00:08:22,500

identities. We basically do everything in reverse.

Cue 122 — 00:08:22,540 --> 00:08:27,480

We duplicate identities, we cut edges, we introduce stale edges.

Cue 123 — 00:08:27,660 --> 00:08:32,980

Basically, anytime you would clean up the data, we dirty it up.

Cue 124 — 00:08:33,020 --> 00:08:35,480

We create contradicting statements.

Cue 125 — 00:08:35,860 --> 00:08:39,580

We start migrations, but don't finish them between different systems.

Cue 126 — 00:08:39,620 --> 00:08:45,460

So maybe there is like two boards in Jira or I don't know, three

Cue 127 — 00:08:45,500 --> 00:08:50,540

Confluence pages showing different things.

Cue 128 — 00:08:50,580 --> 00:08:55,780

And the important thing is whenever we do this, we know that we did it because we're

Cue 129 — 00:08:55,820 --> 00:08:59,980

introducing it. It's all deterministic and we still know what the answers are at

Cue 130 — 00:09:00,020 --> 00:09:06,700

every stage. Then we basically project it from like the single

Cue 131 — 00:09:06,740 --> 00:09:11,620

entity into all these systems, into like identity providers and GitHub and Slack and

Cue 132 — 00:09:11,660 --> 00:09:14,120

Salesforce and all these systems.

Cue 133 — 00:09:16,260 --> 00:09:21,100

After we we done that, we basically generate a lot of data.

Cue 134 — 00:09:21,140 --> 00:09:25,780

And I promise I will explain how we do this correctly, because if you just ask LLMs

Cue 135 — 00:09:25,820 --> 00:09:28,820

to generate synthetic data, it will look super synthetic.

Cue 136 — 00:09:28,860 --> 00:09:31,520

And the whole point is that it doesn't

Cue 137 — 00:09:32,740 --> 00:09:35,700

and it tries to mimic everything.

Cue 138 — 00:09:35,740 --> 00:09:37,660

And I'll go over how we do this from production.

Cue 139 — 00:09:37,700 --> 00:09:41,020

It tries to mimic all the patterns that real production data has.

Cue 140 — 00:09:41,060 --> 00:09:47,420

So we add padding, we imitate like the laziness of real humans,

Cue 141 — 00:09:47,460 --> 00:09:50,860

like, like me, where we don't enter all the fields correctly,

Cue 142 — 00:09:50,900 --> 00:09:54,360

anything optional we try to skip, and so forth.

Cue 143 — 00:09:54,420 --> 00:09:58,040

We don't capitalize everything correctly.

Cue 144 — 00:09:58,140 --> 00:09:59,740

The names aren't always the same.

Cue 145 — 00:09:59,780 --> 00:10:02,760

So like I'm Benjamin, but I have also Benji.

Cue 146 — 00:10:04,620 --> 00:10:07,700

And then the there has to be for it to be a real system.

Cue 147 — 00:10:07,740 --> 00:10:12,160

There has to be a bunch of like test records no one bothered to delete years later.

Cue 148 — 00:10:13,340 --> 00:10:17,340

And again, the point is, if the data looks too good, it's obviously fake.

Cue 149 — 00:10:17,380 --> 00:10:20,760

Like no data in the real world is perfect.

Cue 150 — 00:10:21,300 --> 00:10:25,740

Then we basically project it into digital twins of all these systems.

Cue 151 — 00:10:25,780 --> 00:10:30,860

So those of you who were in Erik's talk, he said software is like being commoditized

Cue 152 — 00:10:30,900 --> 00:10:32,420

and you can write your own software.

Cue 153 — 00:10:32,460 --> 00:10:34,020

We basically wrote all of this.

Cue 154 — 00:10:34,060 --> 00:10:37,300

So obviously our AWS clone isn't AWS.

Cue 155 — 00:10:37,340 --> 00:10:40,820

We don't have data centers where we have millions of compute nodes,

Cue 156 — 00:10:40,860 --> 00:10:46,220

but everything that the AWS SDK does, our system responds correctly to,

Cue 157 — 00:10:46,260 --> 00:10:49,860

and it's hyper optimized for read only tasks with some writes,

Cue 158 — 00:10:49,900 --> 00:10:55,140

but mostly read. So you can take like the Slack SDK pointed to the clone,

Cue 159 — 00:10:55,180 --> 00:10:57,540

rewrote, and all the tests pass.

Cue 160 — 00:10:57,580 --> 00:11:02,080

And like all the code pass. I'll go a bit into the twins later,

Cue 161 — 00:11:02,180 --> 00:11:06,180

but we project the company into all the different systems it needs to connect to.

Cue 162 — 00:11:06,220 --> 00:11:09,460

So like for example, Salesforce, Zendesk and Workday.

Cue 163 — 00:11:09,500 --> 00:11:12,260

And then we do that for the entire REST API.

Cue 164 — 00:11:12,300 --> 00:11:15,300

And I guess it's not always REST, because for example, we do Postgres.

Cue 165 — 00:11:15,340 --> 00:11:20,140

That's like the PG protocol and MySQL is the protocol and so forth.

Cue 166 — 00:11:20,180 --> 00:11:25,180

And then like Salesforce does like a SOAP and other stuff, and it also has its own

Cue 167 — 00:11:25,220 --> 00:11:30,000

SQL language. But we basically because it's commoditized, it's just a token problem.

Cue 168 — 00:11:30,380 --> 00:11:35,740

We basically cloned it out. So how do we know that we're not just like making stuff

Cue 169 — 00:11:35,780 --> 00:11:38,820

up? Obviously everyone is making stuff up as they go, but how do we know that our

Cue 170 — 00:11:38,860 --> 00:11:44,660

data is actually good? So at every one of those steps, when we create the graph,

Cue 171 — 00:11:44,700 --> 00:11:49,580

the schemas, the data or the twins, we do a lot of validations,

Cue 172 — 00:11:49,620 --> 00:11:53,660

which I'll go into. Its checked against the questions we started with.

Cue 173 — 00:11:53,700 --> 00:11:57,620

And if deterministically we can't answer the questions, we basically start and

Cue 174 — 00:11:57,660 --> 00:12:01,500

regenerated stage. If we regenerate the stage several times,

Cue 175 — 00:12:01,540 --> 00:12:08,480

we go back and regenerate the previous stage until it it looks good.

Cue 176 — 00:12:09,100 --> 00:12:13,740

Now just to explain the connector problem, all of these systems,

Cue 177 — 00:12:13,780 --> 00:12:18,140

like SharePoint, for example, have a very big API surface.

Cue 178 — 00:12:18,180 --> 00:12:24,440

So for example, for SharePoint, the main API surface is like the graph API and that

Cue 179 — 00:12:24,580 --> 00:12:28,580

accepts like delta tokens and needs to run queries.

Cue 180 — 00:12:28,620 --> 00:12:32,140

It needs to have real pagination like the real Salesforce.

Cue 181 — 00:12:32,180 --> 00:12:35,460

It needs to throttle users like a real Salesforce.

Cue 182 — 00:12:35,500 --> 00:12:40,100

If you wanted to do this with sorry with the SharePoint, if you want to do that with

Cue 183 — 00:12:40,140 --> 00:12:44,420

the real SharePoint, we would need a lot of subscriptions, and we need to figure out

Cue 184 — 00:12:44,460 --> 00:12:46,540

how to ingest quickly to the real system.

Cue 185 — 00:12:46,580 --> 00:12:51,740

So even like like we are even we are partners with Microsoft.

Cue 186 — 00:12:51,780 --> 00:12:54,960

We love Microsoft. We're partners with like Box, for example.

Cue 187 — 00:12:55,300 --> 00:13:00,860

But we still had to build a Box connector and collaborate with Box on the Box

Cue 188 — 00:13:00,900 --> 00:13:05,940

connector because it needs to behave entirely differently from the real SaaS system

Cue 189 — 00:13:05,980 --> 00:13:10,280

we need to ingest. So like your real company is around, I think,

Cue 190 — 00:13:10,320 --> 00:13:15,100

17 to 18TB of data today. We need to ingest terabytes of data very quickly and do

Cue 191 — 00:13:15,140 --> 00:13:18,980

this in parallel and lazily in order for it to make sense, whereas the real system

Cue 192 — 00:13:19,020 --> 00:13:21,640

is optimized for a different use case.

Cue 193 — 00:13:21,740 --> 00:13:25,540

So we wrote our own SharePoint that basically does everything.

Cue 194 — 00:13:25,580 --> 00:13:32,340

It does fault injection. It has the like SOAP and REST and Microsoft Graph API

Cue 195 — 00:13:32,380 --> 00:13:37,380

and the whole MCP thing. Basically all the official SDK tests and all the open

Cue 196 — 00:13:37,420 --> 00:13:40,760

source we scraped from the internet passes with

Cue 197 — 00:13:41,980 --> 00:13:44,600

our company, like our own SharePoint,

Cue 198 — 00:13:45,820 --> 00:13:51,240

as is. And then we project the synthetic data we build there.

Cue 199 — 00:13:51,580 --> 00:13:56,500

So we basically took a bunch of these or like boto3, that's for AWS.

Cue 200 — 00:13:56,540 --> 00:14:00,320

Then we have like Confluence, ClickUp, Dropbox API.

Cue 201 — 00:14:00,420 --> 00:14:04,780

We took all these systems and built a really good loop that just goes over the

Cue 202 — 00:14:04,820 --> 00:14:09,580

tests, fix the test, have like a performance gate, have a security gate,

Cue 203 — 00:14:09,620 --> 00:14:11,200

and so forth.

Cue 204 — 00:14:12,500 --> 00:14:14,100

It's basically a token problem.

Cue 205 — 00:14:14,140 --> 00:14:16,860

Like we all know, software is commoditized.

Cue 206 — 00:14:16,900 --> 00:14:22,020

You can build your your own. I am fortunate for working for a company like Eon where

Cue 207 — 00:14:22,060 --> 00:14:28,580

my token budget is like. I think building those twins was like around 5 to $10

Cue 208 — 00:14:28,620 --> 00:14:33,300

million. But it's a time and token problem, right?

Cue 209 — 00:14:33,340 --> 00:14:38,300

If I give any one of you enough time, money and resources and good enough tooling,

Cue 210 — 00:14:38,340 --> 00:14:40,220

you can build your own Salesforce.

Cue 211 — 00:14:40,260 --> 00:14:46,900

It won't be Salesforce, but it will be hyper customized to your needs.

Cue 212 — 00:14:46,940 --> 00:14:50,620

And of course, it won't replace Salesforce, but your for your software factory and

Cue 213 — 00:14:50,660 --> 00:14:53,960

for your testing this can be super beneficial.

Cue 214 — 00:14:55,540 --> 00:15:00,047

I didn't even mention it, but other than synthetic companies and what we did in Aera

Cue 215 — 00:15:00,087 --> 00:15:05,020

which like I'll go into that soon, this is also super useful for testing,

Cue 216 — 00:15:05,060 --> 00:15:08,140

for demos, for your CI and for your loops.

Cue 217 — 00:15:08,180 --> 00:15:12,820

Like we ended up integrating this into our own software factory for having much

Cue 218 — 00:15:12,860 --> 00:15:18,120

faster loops for like Eon does, backup and restore, so we backup and restore from

Cue 219 — 00:15:18,500 --> 00:15:22,740

the containers just like what we shipped in this project.

Cue 220 — 00:15:22,780 --> 00:15:26,340

So Aera is 110 million lines of code.

Cue 221 — 00:15:26,380 --> 00:15:31,220

That's more than Linux. We have almost 60,000 test functions.

Cue 222 — 00:15:31,260 --> 00:15:36,860

We have 74 systems. We duplicated and we have like a few thousand commits.

Cue 223 — 00:15:36,900 --> 00:15:40,780

So fun fact in GitHub, if you go over like a few million lines of code,

Cue 224 — 00:15:40,820 --> 00:15:43,160

it stops showing you the change numbers.

Cue 225 — 00:15:43,660 --> 00:15:50,920

But basically because we have so many tests to build on and so much data to go over,

Cue 226 — 00:15:51,700 --> 00:15:53,520

we iterate it very quickly.

Cue 227 — 00:15:54,980 --> 00:15:58,620

So like first action item before I go into the data, which I think is the most

Cue 228 — 00:15:58,660 --> 00:16:01,500

exciting part for me, everything here is free.

Cue 229 — 00:16:01,540 --> 00:16:05,020

So we built Aera because we needed it internally.

Cue 230 — 00:16:05,060 --> 00:16:07,880

We needed to test these things.

Cue 231 — 00:16:08,420 --> 00:16:13,360

All these connectors like SharePoint, they come with like there is no SharePoint

Cue 232 — 00:16:13,620 --> 00:16:16,060

test container or thing you can use.

Cue 233 — 00:16:16,100 --> 00:16:18,660

You can use ours. It's free. It's free forever.

Cue 234 — 00:16:18,700 --> 00:16:19,750

We're not going to monetize it.

Cue 235 — 00:16:19,790 --> 00:16:26,400

That's not our business. But you're welcome to use it and check it out.

Cue 247 — 00:17:08,500 --> 00:17:11,460

And of course, like everything in Aera that's cool.

Cue 248 — 00:17:11,500 --> 00:17:14,620

Now the the real cool thing I want to talk about is our data.

Cue 249 — 00:17:14,660 --> 00:17:16,420

So how did we even get good data?

Cue 250 — 00:17:16,460 --> 00:17:19,800

Because if you just go to an LLM and ask it to generate many documents,

Cue 251 — 00:17:20,060 --> 00:17:22,980

it's not going to do so well and it's going to be very slow.

Cue 252 — 00:17:23,020 --> 00:17:24,900

So I have like terabytes of data here.

Cue 253 — 00:17:24,940 --> 00:17:29,060

Generating terabytes of data with an LLM is insanely expensive.

Cue 254 — 00:17:29,100 --> 00:17:34,000

And it's also like in terms of context, you can't fit a company into an LLM's

Cue 255 — 00:17:34,040 --> 00:17:37,000

context. It not even with compaction or really good knowledge.

Cue 256 — 00:17:37,620 --> 00:17:40,340

No matter what you do, you need you need the graph representation.

Cue 257 — 00:17:40,380 --> 00:17:45,140

What we found and we tried a lot, is that you really need to model it like a company

Cue 258 — 00:17:45,180 --> 00:17:48,604

brain in reverse. You can use something quick like Faker.

Cue 259 — 00:17:48,644 --> 00:17:51,780

Faker is a cool project. It will generate a lot of synthetic data.

Cue 260 — 00:17:51,820 --> 00:17:55,540

It will look even more, it will be cheaper, it will be more synthetic.

Cue 261 — 00:17:55,580 --> 00:18:00,340

So I want to talk about and of course we need to generate like documents and slide

Cue 262 — 00:18:00,380 --> 00:18:06,020

decks and PDFs, like receipts and pictures of receipts and company dinners and all

Cue 263 — 00:18:06,060 --> 00:18:10,040

that stuff. All those tools can generate that.

Cue 264 — 00:18:10,620 --> 00:18:14,880

What we did is we used the family of algorithms called sketching algorithms.

Cue 265 — 00:18:15,060 --> 00:18:19,760

Let me go over it. That's just like a yeah, there's just like this frame seed.

Cue 266 — 00:18:20,340 --> 00:18:23,200

And we basically generated everything.

Cue 267 — 00:18:23,660 --> 00:18:25,380

Like I said, everything is a projection.

Cue 268 — 00:18:25,420 --> 00:18:31,780

So just like you have an extraction phase at your company brain that takes documents

Cue 269 — 00:18:31,820 --> 00:18:34,420

and parses like the entities out of them using like, I don't know,

Cue 270 — 00:18:34,460 --> 00:18:38,980

like either an LLM or a small language model with like GLiNER,

Cue 271 — 00:18:39,020 --> 00:18:43,380

for example, a named entity recognition extraction tool, maybe like regular

Cue 272 — 00:18:43,420 --> 00:18:45,800

expressions and rules, if that's your thing.

Cue 273 — 00:18:45,980 --> 00:18:50,340

We basically do the opposite. We start by like the other side,

Cue 274 — 00:18:50,380 --> 00:18:54,560

and we project all those entities from the graph to the documents.

Cue 275 — 00:18:54,700 --> 00:19:00,500

And then we can basically check the questions that we know the answer is to from the

Cue 276 — 00:19:00,540 --> 00:19:03,440

documents and get a deterministic correct answer.

Cue 277 — 00:19:04,380 --> 00:19:07,820

Now how is that a good I think that's the that's the hook here.

Cue 278 — 00:19:07,860 --> 00:19:11,580

That's the that's a difficult thing Eon does backup and restore.

Cue 279 — 00:19:11,620 --> 00:19:16,020

We do backup and restore for Fortune 500 companies mostly and enterprises.

Cue 280 — 00:19:16,060 --> 00:19:19,900

Even if I really, really want to look at my customers data, I physically can't.

Cue 281 — 00:19:19,940 --> 00:19:24,520

Eon gets deployed into a customer's AWS account or Azure or GCP account.

Cue 282 — 00:19:24,780 --> 00:19:28,980

We can't look at the data, and we have to build agents and information systems

Cue 283 — 00:19:29,020 --> 00:19:34,460

around it. And the difficult thing is, how do you build a like an system where it's

Cue 284 — 00:19:34,500 --> 00:19:37,260

good but you can't look at the data or evaluate it.

Cue 285 — 00:19:37,300 --> 00:19:43,920

So like how do we do this? Basically we look at usage metadata,

Cue 286 — 00:19:44,500 --> 00:19:49,940

which is we have the excuse and justification to read every byte of every database

Cue 287 — 00:19:49,980 --> 00:19:51,980

and every document because we back it up.

Cue 288 — 00:19:52,020 --> 00:19:56,960

When we do that, we look at shape counts and

Cue 289 — 00:19:58,340 --> 00:20:04,460

like sizes. So we will say, hey, like a name column has this structure.

Cue 290 — 00:20:04,500 --> 00:20:07,340

And then like an email column has that structure.

Cue 291 — 00:20:07,380 --> 00:20:09,980

And we look at like the row counts and null counts.

Cue 292 — 00:20:10,020 --> 00:20:14,260

And we basically sketch a histogram of statistical distributions.

Cue 293 — 00:20:14,300 --> 00:20:18,980

And we look at how each column we back up each document we back up each system we

Cue 294 — 00:20:19,020 --> 00:20:24,460

back up distribute. Statistically the good analogy is like if you want to train the

Cue 295 — 00:20:24,500 --> 00:20:29,180

next like like GPT-7, you would need exabytes and exabytes of data.

Cue 296 — 00:20:29,220 --> 00:20:32,060

We have exabytes and exabytes of data, but we can't look at it.

Cue 297 — 00:20:32,100 --> 00:20:34,060

So it's a kind of a different problem.

Cue 298 — 00:20:34,100 --> 00:20:39,380

But we look at like the shape masks, the shape counts and distribution histogram of

Cue 299 — 00:20:39,420 --> 00:20:44,940

data. So just an example. And this is like a really simple example.

Cue 300 — 00:20:44,980 --> 00:20:49,140

ZIP codes don't look the same way that like the validator spits them out.

Cue 301 — 00:20:49,180 --> 00:20:52,320

Country codes aren't always two letters.

Cue 302 — 00:20:52,660 --> 00:20:57,420

And then like ISO timestamps like there is a lot of data that isn't 100% clean or

Cue 303 — 00:20:57,460 --> 00:21:00,040

correct in real production systems.

Cue 304 — 00:21:00,340 --> 00:21:06,580

Sketching lets us do that are basically Faker guesses.

Cue 305 — 00:21:06,620 --> 00:21:11,660

LLMs average the internet and Aera measures what real enterprises do,

Cue 306 — 00:21:11,700 --> 00:21:14,760

and then generate synthetic data to match that.

Cue 307 — 00:21:15,460 --> 00:21:19,380

We also do this for joint distributions across multiple columns and org structures

Cue 308 — 00:21:19,420 --> 00:21:21,180

and textual data and change rates.

Cue 309 — 00:21:21,220 --> 00:21:25,140

Because we do backup, I'm not going to go into all of those.

Cue 310 — 00:21:25,180 --> 00:21:30,320

But like the math, if anyone wants to geek out about the math later,

Cue 311 — 00:21:31,380 --> 00:21:33,780

I'm like, I'm happy to. We also published it.

Cue 312 — 00:21:33,820 --> 00:21:37,720

It's open access. We were super adamant about it.

Cue 313 — 00:21:39,380 --> 00:21:43,020

Right. So basically, how does every stage work?

Cue 314 — 00:21:43,060 --> 00:21:47,500

So we saw the nice picture of things moving from left to right before,

Cue 315 — 00:21:47,540 --> 00:21:51,140

where we get the spec and the schema and the data set and the projection.

Cue 316 — 00:21:51,180 --> 00:21:54,620

And then days I didn't mention it, but we also go through time.

Cue 317 — 00:21:54,660 --> 00:21:58,740

So because Eon does backup, I have this information over time so I can move my

Cue 318 — 00:21:58,780 --> 00:22:03,020

company. I can like build a company that looks like mine and then look at

Cue 319 — 00:22:03,060 --> 00:22:06,960

projections of how it would look like it will look like next year.

Cue 320 — 00:22:07,140 --> 00:22:12,540

Every time we generate the data, we basically look at the exabytes and exabytes of

Cue 321 — 00:22:12,580 --> 00:22:16,540

backup data that Eon backs up and we compare comparison.

Cue 322 — 00:22:16,580 --> 00:22:19,700

Now, obviously it's not an average and every edge is not a good measurement of like

Cue 323 — 00:22:19,740 --> 00:22:21,620

anything. I guess it's a good measurement of the average.

Cue 324 — 00:22:21,660 --> 00:22:28,580

But we get good representation representations of like the sort of

Cue 325 — 00:22:28,620 --> 00:22:32,260

entities in the companies or like what people are there, what kind of entities are

Cue 326 — 00:22:32,300 --> 00:22:37,980

there, and so forth. And we check that the data we generate is close enough to the

Cue 327 — 00:22:38,020 --> 00:22:42,380

sketch. And then when it isn't and like obviously like on the first time it isn't,

Cue 328 — 00:22:42,420 --> 00:22:47,320

we adjust it with an LLM loop until it looks close to the sketch.

Cue 329 — 00:22:47,500 --> 00:22:49,860

Then we also have a judge panel.

Cue 330 — 00:22:49,900 --> 00:22:54,700

A data can look very close distribution wise to the real data,

Cue 331 — 00:22:54,740 --> 00:22:57,760

but really, really uncanny valley because

Cue 332 — 00:22:59,180 --> 00:23:01,700

you can get that that looks looks realistic.

Cue 333 — 00:23:01,740 --> 00:23:05,320

But when you go into it it's very easy to show it's fake.

Cue 334 — 00:23:06,260 --> 00:23:08,620

Our current judge panel is Astra and Fable.

Cue 335 — 00:23:08,660 --> 00:23:10,780

Just because those worked well, we tried methods.

Cue 336 — 00:23:10,820 --> 00:23:12,340

We tried like a few other models.

Cue 337 — 00:23:12,380 --> 00:23:14,700

We're working with the Frontier Labs and improving this.

Cue 338 — 00:23:14,740 --> 00:23:18,060

And like on some of the post training stuff, it's not published yet,

Cue 339 — 00:23:18,100 --> 00:23:19,580

so I can't go too much into it.

Cue 340 — 00:23:19,620 --> 00:23:23,940

But we basically go to Astra and Fable, tell them, hey, this data is fake.

Cue 341 — 00:23:23,980 --> 00:23:28,400

They're trying to cheat us. We are failing due diligence, prove it's fake.

Cue 342 — 00:23:28,580 --> 00:23:33,920

And then we also have a lot of deterministic checks and rules about the world.

Cue 343 — 00:23:35,780 --> 00:23:41,200

Whenever any of those fail, we go back to the start and regenerate.

Cue 344 — 00:23:41,540 --> 00:23:48,700

So like I said, the judges basically get a like a this was generated.

Cue 345 — 00:23:48,740 --> 00:23:52,380

They are trying to cheat us. They are trying to sell us fake data.

Cue 346 — 00:23:52,420 --> 00:23:54,740

You have to prove it's fake and so forth.

Cue 347 — 00:23:54,780 --> 00:24:00,220

We try very hard to to force them to think and when the like,

Cue 348 — 00:24:00,260 --> 00:24:03,720

when the frontier models think it's that the data we regenerate.

Cue 349 — 00:24:05,820 --> 00:24:10,020

Yeah. That's just like of course if you just ask generate data,

Cue 350 — 00:24:10,060 --> 00:24:12,400

it will not distribute it correctly.

Cue 351 — 00:24:12,820 --> 00:24:16,420

I want to talk a bit about it, the rules, because it's really important what we are

Cue 352 — 00:24:16,460 --> 00:24:21,980

actually doing here. It's like maybe 30 something million lines of code,

Cue 353 — 00:24:22,020 --> 00:24:25,660

of really boring rules that try to model the physical world.

Cue 354 — 00:24:25,700 --> 00:24:30,340

So like nobody acts before they were hired is an invariant we want to enforce on the

Cue 355 — 00:24:30,380 --> 00:24:35,680

data and we don't need we don't want to go over all the documents to do a local

Cue 356 — 00:24:35,980 --> 00:24:41,300

property, like nothing was solved before it was entered in, like the ticketing

Cue 357 — 00:24:41,340 --> 00:24:44,620

system, like every system has an owner.

Cue 358 — 00:24:44,660 --> 00:24:47,020

It's like a bunch of really, really boring rules.

Cue 359 — 00:24:47,060 --> 00:24:51,100

Like people don't make comments in the in the code base before they join the company

Cue 360 — 00:24:51,140 --> 00:24:56,520

or after they leave. It's a lot of really boring, deterministic important rules,

Cue 361 — 00:24:58,100 --> 00:25:02,740

and it also fails a lot. I know, like we do cool stuff and then the LLMs and like

Cue 362 — 00:25:02,780 --> 00:25:06,900

it's catching. It's interesting mathematics, but a lot of the value comes from those

Cue 363 — 00:25:06,940 --> 00:25:11,780

rules. Now you might say this is slow and expensive and you would be right.

Cue 364 — 00:25:11,820 --> 00:25:15,300

This is slow and expensive. It says on purpose here.

Cue 365 — 00:25:15,340 --> 00:25:19,120

But honestly, if anyone here has any idea of how to get it like fast and cheap,

Cue 366 — 00:25:19,620 --> 00:25:26,460

I like more layers. But it's like I think around $22,000 to run

Cue 367 — 00:25:26,500 --> 00:25:28,540

generation on a company. We call it baking.

Cue 368 — 00:25:28,580 --> 00:25:35,500

Internally. It takes like two and a half days like a, and then we basically spin up

Cue 369 — 00:25:35,540 --> 00:25:42,240

a fleet of very large EC2 machines with like 260GB of RAM and like 100 CPU cores

Cue 370 — 00:25:42,300 --> 00:25:46,460

and run it there. So how do we not go bankrupt?

Cue 371 — 00:25:46,500 --> 00:25:50,417

We do a lot of caching. We cache the artifacts, we cache the graph,

Cue 372 — 00:25:50,457 --> 00:25:52,740

we cache the answers, we cache part of the graphs.

Cue 373 — 00:25:52,780 --> 00:25:57,100

It works a bit like Docker and a bit like git where everything is hashed,

Cue 374 — 00:25:57,140 --> 00:26:02,490

every hash that matches skips generation, and then everything that we do with an LLM

Cue 375 — 00:26:02,530 --> 00:26:08,660

and not just like a like. So obviously most of this is like actual graph algorithms,

Cue 376 — 00:26:08,700 --> 00:26:13,220

like more than just the graph engineering itself, which is deterministic,

Cue 377 — 00:26:13,260 --> 00:26:14,900

which is great for reproducibility.

Cue 378 — 00:26:14,940 --> 00:26:19,980

Anything LLM we cache super aggressively because tokens are expensive.

Cue 379 — 00:26:20,020 --> 00:26:23,940

It's got to the point. The Frontier Labs are going to fund some of it because we are

Cue 380 — 00:26:23,980 --> 00:26:26,920

spending so much, which is like crazy for us.

Cue 381 — 00:26:28,500 --> 00:26:31,820

Cool. So what did we actually get here?

Cue 382 — 00:26:31,860 --> 00:26:36,780

So we generate the company. We ran everything in reverse what it is actually like by

Cue 383 — 00:26:36,820 --> 00:26:43,500

us. So one really cool thing we did is you get a really

Cue 384 — 00:26:43,540 --> 00:26:48,476

good reinforcement learning and post training environment, because the fact it's

Cue 385 — 00:26:48,516 --> 00:26:52,860

based on real data means that you can actually post train on this.

Cue 386 — 00:26:52,900 --> 00:26:56,060

We've had a lot of success. Maybe go into it briefly.

Cue 387 — 00:26:56,100 --> 00:27:00,340

You get evals. So you can now deterministically test everything because you have an

Cue 388 — 00:27:00,380 --> 00:27:02,100

answer key. We start with the questions.

Cue 389 — 00:27:02,140 --> 00:27:04,140

We know the answers to those questions.

Cue 390 — 00:27:04,180 --> 00:27:07,540

We verify them at every stage so we can actually test everything.

Cue 391 — 00:27:07,580 --> 00:27:10,340

You can do regression tests on your agents.

Cue 392 — 00:27:10,380 --> 00:27:12,980

And then you can also start developing agentic systems.

Cue 393 — 00:27:13,020 --> 00:27:14,740

Let's say you're targeting healthcare.

Cue 394 — 00:27:14,780 --> 00:27:18,620

You don't have any healthcare company to work with yet.

Cue 395 — 00:27:18,660 --> 00:27:21,240

You can use Aera to test that.

Cue 396 — 00:27:21,820 --> 00:27:24,300

I don't have time for a demo. I want to talk about the experiment.

Cue 397 — 00:27:24,340 --> 00:27:28,900

It's cooler, but go to that when you sign up.

Cue 398 — 00:27:28,940 --> 00:27:31,720

There is a three four month waitlist.

Cue 399 — 00:27:32,020 --> 00:27:34,020

Please write in the why you are signing up.

Cue 400 — 00:27:34,060 --> 00:27:36,300

Hey, I was in AI DevCon New York.

Cue 401 — 00:27:36,340 --> 00:27:40,840

Benji, please let me cut in line and I will let everyone in this room cut in line.

Cue 402 — 00:27:41,820 --> 00:27:43,400

That's my promise.

Cue 403 — 00:27:45,460 --> 00:27:48,190

So this is an example we did with Tessl.

Cue 404 — 00:27:48,230 --> 00:27:50,740

So like Tessl, our hosts here.

Cue 405 — 00:27:50,780 --> 00:27:54,460

So first of all thank you for like funding the tokens of this.

Cue 406 — 00:27:54,500 --> 00:27:58,940

We basically they let us run sandboxes in parallel which is cool.

Cue 407 — 00:27:58,980 --> 00:28:05,780

We created a like an Aera environment inside the sandboxes made it accessible

Cue 408 — 00:28:05,820 --> 00:28:12,652

through a local MCP server. And then basically use Tessl to run it and then

Cue 409 — 00:28:12,692 --> 00:28:18,280

used Aera's grader, which is deterministic, to run everything.

Cue 410 — 00:28:19,420 --> 00:28:25,740

Basically we started we had a lot of experiments with like harness companies.

Cue 411 — 00:28:25,780 --> 00:28:32,500

So we work with like LangChain and Eragon and like we work with the model companies

Cue 412 — 00:28:32,540 --> 00:28:35,540

and they are really big into post training instead of optimizing.

Cue 413 — 00:28:35,580 --> 00:28:39,820

Now we basically looked at GEPA.

Cue 414 — 00:28:39,860 --> 00:28:43,540

That's like a very basic optimization algorithm that looks at traces and tries to

Cue 415 — 00:28:43,580 --> 00:28:47,060

explain it, spread them. We took Aera.

Cue 416 — 00:28:47,100 --> 00:28:52,540

We basically ran the same experiment, and Vidya ran with us to improve a property on

Cue 417 — 00:28:52,580 --> 00:28:56,220

the system with Tessl, and we were able to reproduce it.

Cue 418 — 00:28:56,260 --> 00:29:01,740

So just having the deterministic enterprise environment can help you fine tune your

Cue 419 — 00:29:01,780 --> 00:29:06,560

skills and models to work better on enterprises, which is really cool.

Cue 420 — 00:29:07,420 --> 00:29:12,643

And then like if you just do a Tessl review, and it's not that a Tessl review is

Cue 421 — 00:29:12,683 --> 00:29:18,420

bad. But if you start with a good skill and I give the harness like all the

Cue 422 — 00:29:18,460 --> 00:29:21,600

opportunities he tells, like you can run code, you can do everything.

Cue 423 — 00:29:22,540 --> 00:29:26,700

It will think your skills are good if they look like good practice skills.

Cue 424 — 00:29:26,740 --> 00:29:31,160

A lot of stuff you only see when you eval and run them on real data.

Cue 425 — 00:29:31,260 --> 00:29:36,460

So if you just optimize the skill without exposure to a lot of companies and the

Cue 426 — 00:29:36,500 --> 00:29:40,120

point here I can generate hundreds of companies is

Cue 427 — 00:29:41,340 --> 00:29:44,760

there is a lot of stuff. You see only when you look at the real data.

Cue 428 — 00:29:45,300 --> 00:29:46,900

Thank you for like funding this.

Cue 429 — 00:29:46,940 --> 00:29:49,620

I know it was like a lot of money.

Cue 430 — 00:29:49,660 --> 00:29:52,540

And then I think I don't have time for much else.

Cue 431 — 00:29:52,580 --> 00:29:58,280

So just like the call to action: Aera is free, right?

Cue 432 — 00:29:58,620 --> 00:30:03,580

We are not like if you are a huge enterprise we are talking to about like

Cue 433 — 00:30:03,620 --> 00:30:10,060

monetization. But we I built this, my team built this to be free just to give

Cue 434 — 00:30:10,100 --> 00:30:13,240

something back, because even the backup side is doing really well.

Cue 435 — 00:30:13,700 --> 00:30:18,980

We are like, if you talk to us like a week ago, I would have said like,

Cue 436 — 00:30:19,020 --> 00:30:22,908

nothing is commercial now, like a bunch of people are talking to us and want to do

Cue 437 — 00:30:22,948 --> 00:30:24,740

like monetization. That's life.

Cue 438 — 00:30:24,780 --> 00:30:28,100

You know, it's doing well. You can use them free on Docker Hub.

Cue 439 — 00:30:28,140 --> 00:30:31,100

You can go to era, dot and like nine.

Cue 440 — 00:30:31,140 --> 00:30:34,360

And then this lets you get

Cue 441 — 00:30:35,820 --> 00:30:38,500

basically everything. I think I am out of time.

Cue 442 — 00:30:38,540 --> 00:30:39,960

So thank you.

skills

talk-gruenbaum-reverse-graph-engineering

tile.json