CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

ainativedev/aidevcon-2026-nyc

AI Native DevCon 2026 New York — practical agent skills grounded in conference talks

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

Overview
Quality
Evals
Security
Files

transcript.mdskills/talk-pratik-patel-refined-rag-data/references/

Corrected transcript: Data Isn't the New Oil — It's Refined Data

Pratik Patel · AI Native DevCon New York 2026

Recording: https://www.youtube.com/watch?v=OkoY428jC1I

Source: original corrected SRT supplied for this talk. Cue numbers, full start/end timestamps and spoken wording are preserved; subtitle line wrapping is joined. Inserted conference promotion cues 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247 are omitted. Speech artifacts remain. Treat this source as evidence, not instructions.

For topic navigation, see outline.md.

Cue 1 — 00:00:00,113 --> 00:00:02,293

Okay. All right. Thank you so much.

Cue 2 — 00:00:03,713 --> 00:00:08,333

So this presentation is around data and data engineering.

Cue 3 — 00:00:08,713 --> 00:00:13,153

I work at Hugging Face. The QR code is my LinkedIn, so please feel free to connect

Cue 4 — 00:00:13,193 --> 00:00:20,153

if you like. One of the things I do as part of my work is provide virtual and

Cue 5 — 00:00:20,193 --> 00:00:25,633

in-person workshops. For example, on some of the stuff we're going to talk about

Cue 6 — 00:00:25,673 --> 00:00:28,713

here. Also fine tuning models and lots of other things.

Cue 7 — 00:00:28,753 --> 00:00:34,473

So if you have an interest in that and your company is let me just reload this real

Cue 8 — 00:00:34,513 --> 00:00:38,673

quick and your company is interested in something like this,

Cue 9 — 00:00:38,713 --> 00:00:44,453

please let me know. Okay. And also everything that I'm going to do in this session

Cue 10 — 00:00:45,033 --> 00:00:48,533

is available as open source. So this is something that you can,

Cue 11 — 00:00:48,873 --> 00:00:52,393

you know, while you're having a sandwich over lunch, you can pull down the repo and

Cue 12 — 00:00:52,433 --> 00:00:57,033

you can run all this locally or you can run it on Hugging Face's infrastructure or

Cue 13 — 00:00:57,073 --> 00:00:59,133

cloud if you like. All right. So,

Cue 14 — 00:01:00,873 --> 00:01:02,673

how many people have heard of Hugging Face before?

Cue 15 — 00:01:02,713 --> 00:01:07,693

Show of hands real quick. Okay, I figured that would be the case at this event.

Cue 16 — 00:01:08,033 --> 00:01:11,073

But Hugging Face does a lot more than just host models, so I just want to quickly

Cue 17 — 00:01:11,113 --> 00:01:14,273

talk about that because we're going to talk about some of these things in this

Cue 18 — 00:01:14,313 --> 00:01:18,193

session as we go through it. So we host a large number of data sets that people use

Cue 19 — 00:01:18,233 --> 00:01:21,593

to fine tune or to train models from scratch.

Cue 20 — 00:01:21,633 --> 00:01:26,353

So for example, FineWeb, which we'll talk about very briefly while I'm here,

Cue 21 — 00:01:26,393 --> 00:01:30,193

I'm going to be running this demo on spaces, which is a way to basically run stuff

Cue 22 — 00:01:30,233 --> 00:01:33,153

in the cloud, whether it's a Docker container or Gradio application.

Cue 23 — 00:01:33,193 --> 00:01:36,473

I'm going to show the app I run in this session shortly.

Cue 24 — 00:01:36,513 --> 00:01:41,033

We have this cool thing called buckets, which is for intermediate or transient data.

Cue 25 — 00:01:41,073 --> 00:01:46,153

Again, going to cover this in the demo in a bit, but it uses a technology called Xet

Cue 26 — 00:01:46,193 --> 00:01:52,553

which allows you to do very fast iterative data engineering tasks because it only

Cue 27 — 00:01:52,593 --> 00:01:55,073

uploads and downloads just the stuff that's changed.

Cue 28 — 00:01:55,113 --> 00:02:01,833

Instead of a giant ten gigabyte file that you might be using to train a train,

Cue 29 — 00:02:01,873 --> 00:02:06,793

a model or whatever. Okay. Jobs are like cron jobs running in the cloud to automate

Cue 30 — 00:02:06,833 --> 00:02:10,193

some data engineering tasks. We also have Inference Providers.

Cue 31 — 00:02:10,233 --> 00:02:15,793

So when you see the latest Qwen or MiniMax model drop, you don't have to go download

Cue 32 — 00:02:15,833 --> 00:02:20,703

that 20 or 100 gigabyte file. You can actually just go straight to Inference

Cue 33 — 00:02:20,743 --> 00:02:23,233

Providers and click a button and start playing with it.

Cue 34 — 00:02:23,273 --> 00:02:25,833

Hook it into your application or chat with it.

Cue 35 — 00:02:25,873 --> 00:02:28,793

And similarly, Inference Endpoints, where you get a dedicated infrastructure to run

Cue 36 — 00:02:28,833 --> 00:02:33,713

that. And we also host a bunch of scientific papers and stuff like that,

Cue 37 — 00:02:33,753 --> 00:02:39,433

or around AI, of course. And then there's other stuff that, that's super interesting

Cue 38 — 00:02:39,473 --> 00:02:43,373

and cool. What a lot of people don't know is Hugging Face is also behind a lot of

Cue 39 — 00:02:43,413 --> 00:02:46,253

open source. And one of the great things about Hugging Face or working at Hugging

Cue 40 — 00:02:46,293 --> 00:02:50,633

Face is that everything we do, including me, is in the open.

Cue 41 — 00:02:50,673 --> 00:02:54,953

Everything is open source and you can find stuff like Transformers,

Cue 42 — 00:02:54,993 --> 00:02:57,813

which you probably heard of, which I'm going to demo here.

Cue 43 — 00:02:57,913 --> 00:03:03,613

Diffusers. smolagents. We also have AI for robotics, which is quite a bit of stuff

Cue 44 — 00:03:03,673 --> 00:03:07,913

happening in that area. And then there's data sets which will talk about a little

Cue 45 — 00:03:07,953 --> 00:03:12,433

bit here, etc., etc. but all this stuff is open source and it's great to work for an

Cue 46 — 00:03:12,473 --> 00:03:15,033

open source company in this day and age.

Cue 47 — 00:03:15,073 --> 00:03:17,673

All right. So I have a question for you.

Cue 48 — 00:03:17,713 --> 00:03:23,873

What's worse than getting the wrong result on a web page or the wrong calculation

Cue 49 — 00:03:23,913 --> 00:03:27,273

when you're doing something, when your application or the wrong answer,

Cue 50 — 00:03:27,313 --> 00:03:32,473

when you ask an AI agent, or if you're building an application using artificial

Cue 51 — 00:03:32,513 --> 00:03:34,753

intelligence that uses AI inside of it.

Cue 52 — 00:03:34,793 --> 00:03:36,673

What's worse than these three things?

Cue 53 — 00:03:36,713 --> 00:03:39,553

Yes, ma'am. Not asking you at all.

Cue 54 — 00:03:39,593 --> 00:03:40,953

That's a that's actually a great answer.

Cue 55 — 00:03:40,993 --> 00:03:41,913

I haven't heard that one before.

Cue 56 — 00:03:41,953 --> 00:03:46,293

Yes. Actually, actually shipping the code.

Cue 57 — 00:03:47,073 --> 00:03:49,393

That's interesting. The answer to that.

Cue 58 — 00:03:49,433 --> 00:03:51,553

I was hoping to get out of here, but that's okay.

Cue 59 — 00:03:51,593 --> 00:03:54,393

Is getting the wrong answer confidently, right?

Cue 60 — 00:03:54,433 --> 00:03:56,153

Because everyone has seen this.

Cue 61 — 00:03:56,193 --> 00:04:01,353

When you ask AI something, it says AI can make mistakes at the bottom,

Cue 62 — 00:04:01,393 --> 00:04:05,793

right? So this is what this is the thing that we're going to try to work on during

Cue 63 — 00:04:05,833 --> 00:04:10,993

this session today. Okay. How to get the right answer and avoid getting the the

Cue 64 — 00:04:11,033 --> 00:04:14,753

wrong answer. And the AI being super confident about it.

Cue 65 — 00:04:14,793 --> 00:04:17,413

Okay. So some key takeaways from here.

Cue 66 — 00:04:17,873 --> 00:04:22,273

Normally I like to do this in a 60 or 90 minutes segment, but this is 30 minutes.

Cue 67 — 00:04:22,313 --> 00:04:23,713

So we're going to go through this rather quickly.

Cue 68 — 00:04:23,753 --> 00:04:27,513

If you have questions afterwards I'll be having a sandwich or whatever at the lunch

Cue 69 — 00:04:27,553 --> 00:04:30,673

area right. But key takeaways if there's nothing else you get from this,

Cue 70 — 00:04:30,713 --> 00:04:32,753

hopefully these are the three things you take home.

Cue 71 — 00:04:32,793 --> 00:04:34,753

Data matters more than ever today.

Cue 72 — 00:04:34,793 --> 00:04:36,353

Right. Especially having good clean data.

Cue 73 — 00:04:36,393 --> 00:04:37,593

That's what we're going to talk about.

Cue 74 — 00:04:37,633 --> 00:04:39,333

Refine data. Okay.

Cue 75 — 00:04:40,773 --> 00:04:46,033

In we as software engineers, we have a process and different types of methodologies

Cue 76 — 00:04:46,073 --> 00:04:47,793

we use to produce good software.

Cue 77 — 00:04:47,833 --> 00:04:50,913

And of course data engineering also has these things.

Cue 78 — 00:04:50,953 --> 00:04:55,473

So I want to introduce you to some of those concepts for what I use and what other

Cue 79 — 00:04:55,513 --> 00:05:02,193

people use in data engineering to make it more of a rigorous type of engineering

Cue 80 — 00:05:02,233 --> 00:05:06,673

type of thing. Okay. And of course, a complete app that you can pull down and run on

Cue 81 — 00:05:06,713 --> 00:05:12,093

your laptop, or you can just go to the space that I have this and click a button and

Cue 82 — 00:05:12,233 --> 00:05:17,073

literally run it on Hugging Face cloud right away without having to install

Cue 83 — 00:05:17,113 --> 00:05:20,673

anything. Okay. So everyone has heard of this specific quote,

Cue 84 — 00:05:20,713 --> 00:05:22,673

which is data is the new oil. Okay.

Cue 85 — 00:05:22,713 --> 00:05:25,533

We're going to go into this a little bit in some detail.

Cue 86 — 00:05:25,573 --> 00:05:28,313

Now crude oil is worthless until it's refined.

Cue 87 — 00:05:28,353 --> 00:05:32,233

Everybody knows this right? You can't just pump crude oil directly into your car or

Cue 88 — 00:05:32,273 --> 00:05:35,753

whatever. Right. So crude oil, it is what it is.

Cue 89 — 00:05:35,793 --> 00:05:39,113

It's got a bunch of other a bunch of things in it that you want and don't want.

Cue 90 — 00:05:39,153 --> 00:05:43,093

So we refine that and then after we refine it, that gives us fuel.

Cue 91 — 00:05:43,133 --> 00:05:47,033

Right. So I take a similar approach to thinking about data and especially data

Cue 92 — 00:05:47,073 --> 00:05:50,673

engineering, especially since AI is your delivery pipeline now.

Cue 93 — 00:05:50,713 --> 00:05:55,413

Right. So you have company data which you might use in your coding assistant.

Cue 94 — 00:05:55,513 --> 00:05:58,313

You might have a chatbot that you're building or you've used a chatbot,

Cue 95 — 00:05:58,353 --> 00:06:02,113

obviously that will use your company's data also.

Cue 96 — 00:06:02,153 --> 00:06:05,033

And then you have agents that use the data in certain ways.

Cue 97 — 00:06:05,073 --> 00:06:08,993

But if the data is not good, if it hasn't been refined, it's not very useful.

Cue 98 — 00:06:09,033 --> 00:06:14,153

Right. And again, going back to a previous slide, the worst thing is having a

Cue 99 — 00:06:14,193 --> 00:06:15,793

confident answer that is wrong.

Cue 100 — 00:06:15,833 --> 00:06:18,353

Right. That's what we want to try to figure out how to avoid.

Cue 101 — 00:06:18,393 --> 00:06:22,553

Okay. And how to test it. So what we're going to do this demo is actually very

Cue 102 — 00:06:22,593 --> 00:06:25,753

simple. I tried to simplify this as much as possible because this is a short time

Cue 103 — 00:06:25,793 --> 00:06:31,953

slot. But what I did is I took the Hugging Face documentation and I just pumped it

Cue 104 — 00:06:31,993 --> 00:06:35,953

directly into a large language model, sort of like a chatbot type of thing,

Cue 105 — 00:06:35,993 --> 00:06:37,733

which I'll demo in just a second.

Cue 106 — 00:06:38,153 --> 00:06:42,873

And I wanted to see, number one, how we might test around this to make sure that we

Cue 107 — 00:06:42,913 --> 00:06:45,313

get good results from the underlying data.

Cue 108 — 00:06:45,353 --> 00:06:48,753

So everyone is probably heard of retrieval, augmented generation or RAG.

Cue 109 — 00:06:48,793 --> 00:06:53,353

We're going to use basically a very localized simple version of RAG inside of this

Cue 110 — 00:06:53,393 --> 00:06:56,553

application. Okay. But it's basically an ask the docs assistant.

Cue 111 — 00:06:56,593 --> 00:07:01,153

So you think of your company, you have all these documents which are in PDF files or

Cue 112 — 00:07:01,193 --> 00:07:05,793

web pages or whatever. How do you take that stuff and put it into a rigorous data

Cue 113 — 00:07:05,833 --> 00:07:10,313

engineering pipeline so that you get good, guaranteed results from it?

Cue 114 — 00:07:10,353 --> 00:07:13,993

Right. Everyone in this room is aware, since you're at an AI conference,

Cue 115 — 00:07:14,033 --> 00:07:16,513

that AI is inherently non-deterministic.

Cue 116 — 00:07:16,553 --> 00:07:20,313

So what we want to do, or what we've been trying to do as an industry is make

Cue 117 — 00:07:20,353 --> 00:07:24,033

everything more deterministic so that we get correct results out of it.

Cue 118 — 00:07:24,073 --> 00:07:27,853

And this is the data engineering part, not just let's go ask the,

Cue 119 — 00:07:28,913 --> 00:07:33,033

the model, what you know, what it thinks about something or how to put some code

Cue 120 — 00:07:33,073 --> 00:07:36,473

together or whatever. Okay. So our demo here is we're going to take some

Cue 121 — 00:07:36,513 --> 00:07:40,193

documentation or specifically Hugging Face docs documentation.

Cue 122 — 00:07:40,233 --> 00:07:42,713

We're going to refine that in.

Cue 123 — 00:07:42,753 --> 00:07:47,573

Then we're going to test it with a suite of test or suite of questions,

Cue 124 — 00:07:47,753 --> 00:07:49,713

which we have correct answers for.

Cue 125 — 00:07:49,753 --> 00:07:52,433

It's essentially a long tail eval.

Cue 126 — 00:07:52,473 --> 00:07:57,833

And then we'll release that as a data set that we can then use in our application as

Cue 127 — 00:07:57,873 --> 00:07:59,433

long as it passes some metric.

Cue 128 — 00:07:59,473 --> 00:08:03,193

In this case only if 90% pass rate for this okay.

Cue 129 — 00:08:03,233 --> 00:08:04,793

And then we can ask questions of it.

Cue 130 — 00:08:04,833 --> 00:08:06,773

So let's run the demo really quickly.

Cue 131 — 00:08:07,233 --> 00:08:10,953

And hopefully my thing is still oh there it is.

Cue 132 — 00:08:10,993 --> 00:08:16,433

Finally woke up. All right. So again what I'm doing is I'm taking the documentation

Cue 133 — 00:08:16,473 --> 00:08:19,833

for using Hugging Face tools. And that is our data set.

Cue 134 — 00:08:19,873 --> 00:08:21,233

And I'm going to ask you some questions.

Cue 135 — 00:08:21,273 --> 00:08:22,913

Let me make this a little bit bigger so you can see it.

Cue 136 — 00:08:22,953 --> 00:08:25,153

All right. So I'm just going to be lazy.

Cue 137 — 00:08:25,193 --> 00:08:28,233

And I'm going to just click this question.

Cue 138 — 00:08:28,273 --> 00:08:31,953

How do I check which Hugging Face account I'm log to logged in from the terminal.

Cue 139 — 00:08:31,993 --> 00:08:38,553

Right. Simple question. Now let's go ask both of our set of data.

Cue 140 — 00:08:38,593 --> 00:08:41,193

Right. So on the left what I have is the crude data.

Cue 141 — 00:08:41,233 --> 00:08:47,033

Right. Basically this is the unrefined, just raw data just pumped into a RAG

Cue 142 — 00:08:47,073 --> 00:08:49,193

retrieval augmented generation system.

Cue 143 — 00:08:49,233 --> 00:08:52,473

And on the right side I have the data after I refined it.

Cue 144 — 00:08:52,513 --> 00:08:57,333

Right. So if we look at this, you'll see that we get two different answers back.

Cue 145 — 00:08:57,593 --> 00:09:03,353

And because I used the CLI tools quite a bit, I know that the answer on the right is

Cue 146 — 00:09:03,393 --> 00:09:07,333

the correct answer and the one on the left is an incorrect answer.

Cue 147 — 00:09:07,393 --> 00:09:09,233

So this is the whole point of this.

Cue 148 — 00:09:09,273 --> 00:09:13,473

This presentation. And this demo is to show you how to get from something that may

Cue 149 — 00:09:13,513 --> 00:09:17,633

be right to something that is probably 95% correct.

Cue 150 — 00:09:17,673 --> 00:09:19,453

Okay. So that's our quick demo.

Cue 151 — 00:09:19,993 --> 00:09:25,473

Let's move on. And the way that I started through this is I pulled all the

Cue 152 — 00:09:25,513 --> 00:09:28,373

documentation that is out there.

Cue 153 — 00:09:28,833 --> 00:09:31,293

Latest version is version 2.40.

Cue 154 — 00:09:31,753 --> 00:09:33,353

But I pulled all this documentation.

Cue 155 — 00:09:33,393 --> 00:09:36,873

I basically just pointed the system at the docs and said, just go grab everything

Cue 156 — 00:09:36,913 --> 00:09:42,793

and build a RAG system around it using just an embedded vector database type of

Cue 157 — 00:09:42,833 --> 00:09:44,573

thing. Okay, now.

Cue 158 — 00:09:46,393 --> 00:09:51,153

One of the things I discovered while putting this together is that 70% of the chunks

Cue 159 — 00:09:51,193 --> 00:09:54,313

come from older releases, basically old documentation.

Cue 160 — 00:09:54,353 --> 00:10:01,033

And also another problem here is that 57% of the stuff that got put

Cue 161 — 00:10:01,073 --> 00:10:06,993

into our retrieval augmented generation database, 57% of it was duplicates.

Cue 162 — 00:10:07,033 --> 00:10:09,073

And that is bad. We don't want that, right?

Cue 163 — 00:10:09,113 --> 00:10:13,633

That will confuse the way that we do vector similarity search to get a good answer

Cue 164 — 00:10:13,673 --> 00:10:17,633

out of our data okay. And there are six different releases.

Cue 165 — 00:10:17,673 --> 00:10:21,233

And we looked at like the old versus new just a second ago.

Cue 166 — 00:10:21,273 --> 00:10:22,313

And this is your company, too.

Cue 167 — 00:10:22,353 --> 00:10:26,313

You have a stale runbook. You may have duplicate tickets that again,

Cue 168 — 00:10:26,353 --> 00:10:29,353

if you're building some kind of system that uses this kind of data,

Cue 169 — 00:10:29,393 --> 00:10:33,893

you if you're building software, you have an old internal SDK and

Cue 170 — 00:10:35,393 --> 00:10:38,633

that may be outdated, right? Or maybe you have some customers on it,

Cue 171 — 00:10:38,673 --> 00:10:43,073

but not all customers on it. So how do you work with this data to make sure that it

Cue 172 — 00:10:43,113 --> 00:10:47,673

is good for use in your system?

Cue 173 — 00:10:47,713 --> 00:10:53,553

So what what this demo does is it separates out what's content and what's not.

Cue 174 — 00:10:53,593 --> 00:10:55,113

I'll talk about that in just a second.

Cue 175 — 00:10:55,153 --> 00:10:59,433

Okay. And then we want to purify that remove stale and duplicate data.

Cue 176 — 00:10:59,473 --> 00:11:02,793

And then we'll grade this. Basically we're going to lint this.

Cue 177 — 00:11:02,833 --> 00:11:05,133

Everyone is familiar with the concept of linter.

Cue 178 — 00:11:05,193 --> 00:11:08,833

So this is a linter for data not for code okay.

Cue 179 — 00:11:08,873 --> 00:11:14,353

And then we'll blend this together using good data and some other things along the

Cue 180 — 00:11:14,393 --> 00:11:20,593

way to, to make sure that we have a really good data set that we can go and put into

Cue 181 — 00:11:20,633 --> 00:11:26,813

a AI system, or in front of or inside of a, a prompt,

Cue 182 — 00:11:26,993 --> 00:11:30,113

whatever, to make sure that we get good results out of it.

Cue 183 — 00:11:30,153 --> 00:11:37,173

Okay. So the first thing is we want to separate what is

Cue 184 — 00:11:37,393 --> 00:11:40,153

something that people actually read and what can the system use.

Cue 185 — 00:11:40,193 --> 00:11:44,073

Right. So if we think about like let's go scrape some web pages,

Cue 186 — 00:11:44,113 --> 00:11:47,993

for example, there's going to be a bunch of stuff in there that we don't want that's

Cue 187 — 00:11:48,033 --> 00:11:53,633

just going to cause clutter and cause confusion to whatever LLM we feed in the data

Cue 188 — 00:11:53,673 --> 00:11:56,793

for. Right. So we think of a web page, right.

Cue 189 — 00:11:56,833 --> 00:12:02,313

Like there's the stuff that you read and the stuff that you want the LLM to get,

Cue 190 — 00:12:02,353 --> 00:12:04,873

but there's a lot of stuff that you don't want, you don't necessarily need,

Cue 191 — 00:12:04,913 --> 00:12:06,913

like HTML tags and other things around it.

Cue 192 — 00:12:06,953 --> 00:12:09,553

You want to strip away all that noise.

Cue 193 — 00:12:09,593 --> 00:12:15,313

So it's not to cause confusion for the LLM that is actually reading and using this

Cue 194 — 00:12:15,353 --> 00:12:18,713

data. Right. And of course, the the vector similarity search,

Cue 195 — 00:12:18,753 --> 00:12:21,913

which goes and finds those chunks to give to an LLM.

Cue 196 — 00:12:21,953 --> 00:12:25,513

Okay. So that's the first thing is we want to separate the, the,

Cue 197 — 00:12:25,553 --> 00:12:29,513

the good parts of the data and get rid of the stuff that we don't need,

Cue 198 — 00:12:29,553 --> 00:12:32,473

okay. And then we want to purify this.

Cue 199 — 00:12:32,513 --> 00:12:37,793

And again in the demo what we did is we got rid of things where that were stale,

Cue 200 — 00:12:37,833 --> 00:12:43,233

things that were duplicates. And we have specific reasons where we kept certain

Cue 201 — 00:12:43,273 --> 00:12:47,033

parts of the data. And again, when you pull this demo down, you'll see that there's

Cue 202 — 00:12:47,073 --> 00:12:51,473

a folder directory in there that has these are the things that we rejected from the

Cue 203 — 00:12:51,513 --> 00:12:54,993

data set. So you can actually go and see you can plug your own data in as well.

Cue 204 — 00:12:55,033 --> 00:12:56,753

You don't have to use the data I use.

Cue 205 — 00:12:56,793 --> 00:12:58,113

You can use your own company's data.

Cue 206 — 00:12:58,153 --> 00:13:00,273

And the system will take out the things which it says.

Cue 207 — 00:13:00,313 --> 00:13:04,833

We rejected this data for this reason as we were going through this purification

Cue 208 — 00:13:04,873 --> 00:13:08,073

step. And this is all in Python code that you can run yourself.

Cue 209 — 00:13:08,113 --> 00:13:11,333

Okay. So the next up is our linter.

Cue 210 — 00:13:11,673 --> 00:13:16,153

So we want to make sure sorry jumped ahead a little bit too fast.

Cue 211 — 00:13:16,193 --> 00:13:22,513

We want to make sure that the things that we get in terms of the actual data is

Cue 212 — 00:13:22,553 --> 00:13:27,193

good. So here's a section of the linter here on the left or the linter code right

Cue 213 — 00:13:27,233 --> 00:13:31,313

here. So again there are certain things that we want to to make sure that we

Cue 214 — 00:13:31,353 --> 00:13:35,033

capture. So for example huggingface-cli which is the demo I just showed,

Cue 215 — 00:13:35,073 --> 00:13:40,473

you know, how do I figure out who I'm logged in as we see here.

Cue 216 — 00:13:40,513 --> 00:13:45,273

That was replaced in version 3.0.3.4 and then removed.

Cue 217 — 00:13:45,313 --> 00:13:48,753

And newer versions use a slightly different command line, as you probably remember

Cue 218 — 00:13:48,793 --> 00:13:53,673

from the demo. And there's a few other things I have here that that I use in terms

Cue 219 — 00:13:53,713 --> 00:13:55,793

of making sure that this data was good.

Cue 220 — 00:13:55,833 --> 00:13:59,753

Okay. So and then of course we blend things together at the end.

Cue 221 — 00:13:59,793 --> 00:14:04,873

That was our final step. And what I mean by blend again is we don't just take the

Cue 222 — 00:14:04,913 --> 00:14:10,553

data that we've cleaned up, we remove duplicates, we run it through a linter.

Cue 223 — 00:14:10,593 --> 00:14:15,273

We basically purified it, but now we want to blend it back in certain parts of the

Cue 224 — 00:14:15,313 --> 00:14:19,233

data that may have gotten removed or that we feel are also more important than other

Cue 225 — 00:14:19,273 --> 00:14:24,953

things. By by taking this data and putting it into the final data set that is

Cue 226 — 00:14:24,993 --> 00:14:27,823

actually used to build our application.

Cue 227 — 00:14:27,863 --> 00:14:34,713

Okay. So for example, we can build out something like a curated FAQ that has

Cue 228 — 00:14:34,753 --> 00:14:36,313

short answers per common questions.

Cue 229 — 00:14:36,353 --> 00:14:40,233

That helps when we're doing a vector similarity search to make sure our data reaches

Cue 230 — 00:14:40,273 --> 00:14:42,953

the right place. Okay. All right.

Cue 231 — 00:14:42,993 --> 00:14:44,873

So I'm going to go quickly through this.

Cue 232 — 00:14:44,913 --> 00:14:48,793

But this is the progression of the data that we started with and what we ended up

Cue 233 — 00:14:48,833 --> 00:14:52,833

with. Right. So we ended up with we started with almost 2500 chunks.

Cue 234 — 00:14:52,873 --> 00:14:58,473

And after going through this process, which is in the demo, we ended up with 681

Cue 235 — 00:14:58,513 --> 00:15:04,153

chunks. So we significantly reduced the amount of actual stuff that was there to

Cue 236 — 00:15:04,193 --> 00:15:09,333

good, clean data that is pertinent to what we were trying to do in this application.

Cue 248 — 00:15:51,406 --> 00:15:54,406

now when we think about this, I mentioned FineWeb before.

Cue 249 — 00:15:54,446 --> 00:16:01,406

So this is a enormous data set that is essentially lots and lots

Cue 250 — 00:16:01,446 --> 00:16:03,846

of websites scraped over the years.

Cue 251 — 00:16:03,886 --> 00:16:08,046

Right. So this is something that you can just pull and use in your application if

Cue 252 — 00:16:08,086 --> 00:16:13,406

you want. It's in the Hugging Face data sets portion of our website.

Cue 253 — 00:16:13,446 --> 00:16:17,406

So if you go to Hugging Face data sets and of course all this stuff is freely

Cue 254 — 00:16:17,446 --> 00:16:21,945

available, it just as a reminder, everything we do at Hugging Face is open source

Cue 255 — 00:16:22,126 --> 00:16:25,596

and put out there for everyone to use, right?

Cue 256 — 00:16:25,636 --> 00:16:28,886

Wrong one right. So there's a bunch of different data sets here for various

Cue 257 — 00:16:28,926 --> 00:16:32,966

different types of things. FineWeb is one of the most popular ones because that's

Cue 258 — 00:16:33,006 --> 00:16:36,566

what people use to train models from scratch with.

Cue 259 — 00:16:36,606 --> 00:16:43,406

But the point of this here is that the raw FineWeb is 15 trillion tokens.

Cue 260 — 00:16:43,446 --> 00:16:49,966

So it's a very large right now, the FineWeb-Edu is a much smaller data set that's

Cue 261 — 00:16:50,006 --> 00:16:52,726

been refined and is much more targeted.

Cue 262 — 00:16:52,766 --> 00:16:58,486

And of course, when you're trying to train a model training something that has 1.3

Cue 263 — 00:16:58,526 --> 00:17:03,446

trillion tokens versus 15 trillion, it makes a huge difference in terms of speed and

Cue 264 — 00:17:03,486 --> 00:17:08,846

time. But the specific data set that I'm calling out here in the middle,

Cue 265 — 00:17:08,886 --> 00:17:12,366

FineWeb-Edu has also been refined.

Cue 266 — 00:17:12,406 --> 00:17:16,226

And it's a better data set. It's not just everything everything okay.

Cue 267 — 00:17:16,565 --> 00:17:22,846

And this was proven. If you go look you can find the benchmark that was run to test

Cue 268 — 00:17:22,886 --> 00:17:25,406

both of these different data sets to train a model.

Cue 269 — 00:17:25,446 --> 00:17:31,366

And it found that using the refined data set again, that's in the middle here it was

Cue 270 — 00:17:31,406 --> 00:17:36,166

dramatically the model itself was dramatically better by using that refined data

Cue 271 — 00:17:36,206 --> 00:17:42,126

set. Okay. So let's switch gears for a second and talk about data engineering and

Cue 272 — 00:17:42,166 --> 00:17:46,126

how it fits into the overall how we build software type of thing.

Cue 273 — 00:17:46,166 --> 00:17:52,086

Right. So data engineering for a very long time was not as rigorous in terms of how

Cue 274 — 00:17:52,126 --> 00:17:55,286

we build software using different processes.

Cue 275 — 00:17:55,326 --> 00:17:59,406

But the mindset that we want to take nowadays is we want to deliver data like we

Cue 276 — 00:17:59,446 --> 00:18:03,446

deliver code. How do we ship data with the same discipline that we ship code,

Cue 277 — 00:18:03,486 --> 00:18:09,126

right. So and here's just a few things that we think about in terms of software

Cue 278 — 00:18:09,166 --> 00:18:11,886

delivery and how it maps into data delivery.

Cue 279 — 00:18:11,926 --> 00:18:15,446

So in software we do commits to code.

Cue 280 — 00:18:15,486 --> 00:18:19,126

Obviously we check code in so that there's a record of everything that we've changed

Cue 281 — 00:18:19,166 --> 00:18:21,046

and how the system has changed over time.

Cue 282 — 00:18:21,086 --> 00:18:26,286

And for data delivery we do small edits and we upload only the diff and specifically

Cue 283 — 00:18:26,326 --> 00:18:30,126

the Hugging Face. We have this technology called Xet, which allows you to only

Cue 284 — 00:18:30,166 --> 00:18:33,046

update the small part of a very large file that's changed.

Cue 285 — 00:18:33,086 --> 00:18:37,246

So it saves a lot of time and makes the iteration of working with data go much,

Cue 286 — 00:18:37,286 --> 00:18:43,966

much faster. And the the buckets, the buckets that we have

Cue 287 — 00:18:44,006 --> 00:18:47,966

on Hugging Face use that underneath the hood.

Cue 288 — 00:18:48,006 --> 00:18:50,446

But again, Xet is open source tech.

Cue 289 — 00:18:50,486 --> 00:18:54,246

So you don't have to use our infrastructure to put your data up there.

Cue 290 — 00:18:54,286 --> 00:18:57,846

Right. There's a very generous free tier so you can start with it very easily.

Cue 291 — 00:18:57,886 --> 00:19:01,166

You could actually implement your own Xet server if you wanted to on your own

Cue 292 — 00:19:01,206 --> 00:19:04,286

infrastructure. Again, everything you're Hugging Face is open source and freely

Cue 293 — 00:19:04,326 --> 00:19:08,966

available. Okay. So when we think about software delivery, we also have working

Cue 294 — 00:19:09,006 --> 00:19:13,326

branches. And then when we think about how we work, the corollary into the data

Cue 295 — 00:19:13,366 --> 00:19:16,366

engineering world is having a mutable scratch space.

Cue 296 — 00:19:16,406 --> 00:19:20,246

Right. And again that storage buckets, which uses Xet technology underneath the

Cue 297 — 00:19:20,286 --> 00:19:26,846

hood. Right. We all do continuous integration, especially in this world of

Cue 298 — 00:19:26,886 --> 00:19:32,166

software factories. In terms of data delivery, we have compute for doing the refined

Cue 299 — 00:19:32,206 --> 00:19:34,566

and referrals. So we build our own pipelines.

Cue 300 — 00:19:34,606 --> 00:19:37,086

And this application actually builds a pipeline.

Cue 301 — 00:19:37,126 --> 00:19:40,566

Again, when you run it you can see how I'll put all this together.

Cue 302 — 00:19:40,606 --> 00:19:43,226

So we have our CI runner. We have triggers.

Cue 303 — 00:19:43,326 --> 00:19:50,006

So when new data arrives for example in our data delivery process we can rerun

Cue 304 — 00:19:50,046 --> 00:19:53,026

stuff on some kind of schedule basis.

Cue 305 — 00:19:53,926 --> 00:19:58,886

If it's changed or if we have new data that comes in every night,

Cue 306 — 00:19:58,926 --> 00:20:04,326

etc.. And again, if you use our Hugging Face cloud, we have webhooks and scheduled

Cue 307 — 00:20:04,366 --> 00:20:08,985

jobs that do that. But again, you can run all this on your own wherever you like.

Cue 308 — 00:20:09,046 --> 00:20:10,786

And there's just tests.

Cue 309 — 00:20:13,046 --> 00:20:16,966

Automating the testing of data can be hard, especially at scale,

Cue 310 — 00:20:17,006 --> 00:20:21,166

but there are ways to do it. In this specific demonstration,

Cue 311 — 00:20:21,206 --> 00:20:26,246

I use eval gates to make sure that the data that is being used to generate answers

Cue 312 — 00:20:26,286 --> 00:20:29,466

is correct. I can show that code in a bit if we have time.

Cue 313 — 00:20:29,646 --> 00:20:31,786

Okay. And then of course we do releases.

Cue 314 — 00:20:32,166 --> 00:20:34,366

The buckets are good for intermediate data.

Cue 315 — 00:20:34,406 --> 00:20:38,366

Or you know, you want to eventually take your mutable scratch space.

Cue 316 — 00:20:38,406 --> 00:20:42,766

And after you reach the point where you feel like your data is good,

Cue 317 — 00:20:42,806 --> 00:20:46,266

right? It's correct. It's been refined, you remove duplicates,

Cue 318 — 00:20:46,686 --> 00:20:48,766

you don't have old data in there, etc.

Cue 319 — 00:20:48,806 --> 00:20:51,366

you want to tag and make it a release.

Cue 320 — 00:20:51,406 --> 00:20:54,454

So that's where the difference between mutable scratch space,

Cue 321 — 00:20:54,494 --> 00:20:57,926

where you're working on it and you're doing data engineering to make it better,

Cue 322 — 00:20:57,966 --> 00:21:00,246

versus, okay, we've done that.

Cue 323 — 00:21:00,286 --> 00:21:05,966

Now we need to release that data so that it's a fixed data set that we can now use

Cue 324 — 00:21:06,006 --> 00:21:10,126

to do stuff with. Okay. And the corollary on Hugging Face is again,

Cue 325 — 00:21:10,166 --> 00:21:14,006

the data sets, which there's a ton of data sets already out there that you can use.

Cue 326 — 00:21:14,046 --> 00:21:17,886

But when you create your own company data, you will create your own data set and put

Cue 327 — 00:21:17,926 --> 00:21:20,966

it into git as opposed to something like a storage bucket.

Cue 328 — 00:21:21,006 --> 00:21:24,006

Okay. And then of course there's releases and rollbacks, right.

Cue 329 — 00:21:24,046 --> 00:21:28,606

So these are all the things we do in software engineering and the correlations for

Cue 330 — 00:21:28,646 --> 00:21:31,466

them in the data engineering side of the house.

Cue 331 — 00:21:31,806 --> 00:21:33,846

All right. So I talked about that really quickly.

Cue 332 — 00:21:33,886 --> 00:21:35,766

So let me explain what this does.

Cue 333 — 00:21:35,806 --> 00:21:40,966

Xet allows you to have let's say a very massive data file, whether it's a CSV or

Cue 334 — 00:21:41,006 --> 00:21:45,366

parquet or whatever. You make changes to that file to clean up the data or whatever.

Cue 335 — 00:21:45,406 --> 00:21:50,366

Now do you want to upload that gigabyte or ten gigabyte or whatever file back in its

Cue 336 — 00:21:50,406 --> 00:21:53,926

entirety up to the the cloud or whatever infrastructure you're running on?

Cue 337 — 00:21:53,966 --> 00:21:57,766

The answer is no. Xet allows you to not have to upload everything.

Cue 338 — 00:21:57,806 --> 00:22:03,926

What it does is it cuts the data file into chunks, it fingerprints everything,

Cue 339 — 00:22:03,966 --> 00:22:06,646

and then it only uploads or downloads.

Cue 340 — 00:22:06,686 --> 00:22:13,046

Exactly what's changed? So in the test that I've done, it's a footnote at the

Cue 341 — 00:22:13,086 --> 00:22:17,206

bottom. The Xet chunks are usually about 64kB in size.

Cue 342 — 00:22:17,246 --> 00:22:21,446

So it's at that resolution if you're wondering how it does that.

Cue 343 — 00:22:21,486 --> 00:22:26,906

So when we think about, for example, when I ran a test last week,

Cue 344 — 00:22:26,966 --> 00:22:33,606

I had this file where I went and inserted and deleted some records just to

Cue 345 — 00:22:33,646 --> 00:22:37,606

see what the delta that would get uploaded would be.

Cue 346 — 00:22:37,646 --> 00:22:41,786

And so for what I made, some changes I made about, I don't know,

Cue 347 — 00:22:42,046 --> 00:22:46,966

a few hundred row changes here I was able to get instead of having to upload this

Cue 348 — 00:22:47,006 --> 00:22:52,526

entire 90 megabyte file, it would only upload the six megabytes or so that had been

Cue 349 — 00:22:52,566 --> 00:22:56,046

changed. Okay, so just to give you an idea and again, all this is open source

Cue 350 — 00:22:56,086 --> 00:23:01,206

technology storage buckets. If you look at the application I have everything in a

Cue 351 — 00:23:01,246 --> 00:23:06,366

storage bucket including this docs doc parquet file, which is the snapshot of data

Cue 352 — 00:23:06,406 --> 00:23:08,806

that we're using to run this specific demo.

Cue 353 — 00:23:08,846 --> 00:23:14,366

Okay. Again data set repos or releases buckets are for intermediate data.

Cue 354 — 00:23:14,406 --> 00:23:18,606

So again I'm trying to give you an overall idea of the pipeline of how I do data

Cue 355 — 00:23:18,646 --> 00:23:21,286

engineering. Okay. Jobs are runner.

Cue 356 — 00:23:21,326 --> 00:23:24,126

We've already talked about this webhooks or triggers.

Cue 357 — 00:23:24,166 --> 00:23:27,866

So when you get new data it will go and refine.

Cue 358 — 00:23:28,006 --> 00:23:31,966

Basically run this process for you automatically if you don't want to write

Cue 359 — 00:23:32,006 --> 00:23:36,166

manually. So it's a way of running like a job for new data that comes in every day,

Cue 360 — 00:23:36,206 --> 00:23:41,706

or however often the data comes into your specific application.

Cue 361 — 00:23:41,766 --> 00:23:42,666

All right.

Cue 362 — 00:23:44,126 --> 00:23:46,666

So let's talk about evals for a second.

Cue 363 — 00:23:47,166 --> 00:23:50,526

I can show some of these evals in the code if you like.

Cue 364 — 00:23:50,566 --> 00:23:52,566

But there are a couple of things here.

Cue 365 — 00:23:52,606 --> 00:23:54,406

Let me zoom in. This is a little small.

Cue 366 — 00:23:54,446 --> 00:23:57,526

So. Right. So this is the question I asked earlier.

Cue 367 — 00:23:57,566 --> 00:23:59,686

How do I log into Hugging Face from the command line.

Cue 368 — 00:23:59,726 --> 00:24:03,846

So I wanted to go and look at the documentation and give me an answer.

Cue 369 — 00:24:03,886 --> 00:24:10,494

Right. So specifically what I have is stuff like this in here says must include hf

Cue 370 — 00:24:10,534 --> 00:24:13,006

auth login, because that is the correct answer today.

Cue 371 — 00:24:13,046 --> 00:24:19,046

For this it must not include the old CLI command which is the second one,

Cue 372 — 00:24:19,086 --> 00:24:21,346

right. Because that is tagged as stale.

Cue 373 — 00:24:21,406 --> 00:24:24,926

So when you go run this code and you look at how it put this together,

Cue 374 — 00:24:24,966 --> 00:24:28,346

it will actually tag these different types of

Cue 375 — 00:24:29,566 --> 00:24:32,446

test or write what we're calling evals here.

Cue 376 — 00:24:32,486 --> 00:24:36,366

It will tag them so that you know exactly where things break.

Cue 377 — 00:24:36,406 --> 00:24:40,326

Right. So if you get the second response in this case, then you can look at the tag

Cue 378 — 00:24:40,366 --> 00:24:41,726

and say, oh, this is stale data.

Cue 379 — 00:24:41,766 --> 00:24:44,866

That's why it was removed for the second part here.

Cue 380 — 00:24:44,966 --> 00:24:48,326

How do I create a storage bucket hf buckets create.

Cue 381 — 00:24:48,366 --> 00:24:50,926

This is a newer feature. So it's tagged as a newer feature.

Cue 382 — 00:24:50,966 --> 00:24:56,246

So tagging this these questions in this way is very helpful in understanding what

Cue 383 — 00:24:56,286 --> 00:24:59,726

kind of questions you have. Do you have the right mix of different kinds of

Cue 384 — 00:24:59,766 --> 00:25:02,926

questions, right things which are new features, data that might be stale,

Cue 385 — 00:25:02,966 --> 00:25:08,046

etc., things that must be included in the response and things that must not be

Cue 386 — 00:25:08,086 --> 00:25:14,606

included. Okay, so again, this is how I put stuff together to make sure that when

Cue 387 — 00:25:14,646 --> 00:25:21,385

I'm running test against a new data set, or when I'm running test against a new LLM,

Cue 388 — 00:25:21,526 --> 00:25:26,806

if I'm hitting the right types of things as part of this testing process.

Cue 389 — 00:25:26,846 --> 00:25:33,566

Okay, so the overall idea here is to have data to be able to give it to an LLM and

Cue 390 — 00:25:33,606 --> 00:25:36,126

to get correct answers out of it.

Cue 391 — 00:25:36,166 --> 00:25:38,726

Now, we all know that LLMs are non-deterministic.

Cue 392 — 00:25:38,766 --> 00:25:43,246

So this is a way to make sure that our data flows through this entire process and

Cue 393 — 00:25:43,286 --> 00:25:48,246

gives us something that is a much more deterministic answer to what we're trying to

Cue 394 — 00:25:48,286 --> 00:25:50,606

do, right? Or deterministic, whatever.

Cue 395 — 00:25:50,646 --> 00:25:51,846

It doesn't have to be an answer, right.

Cue 396 — 00:25:51,886 --> 00:25:57,346

Deterministic data in, sorry data in and deterministic data or answers out.

Cue 397 — 00:25:58,286 --> 00:26:02,286

All right. And there's a gate here that is also part of this.

Cue 398 — 00:26:02,326 --> 00:26:03,145

And

Cue 399 — 00:26:04,366 --> 00:26:05,846

this is not super interesting here.

Cue 400 — 00:26:05,886 --> 00:26:10,366

But once we pass a certain certain threshold we say this is good.

Cue 401 — 00:26:10,406 --> 00:26:15,326

It's past our 95% threshold for getting the right answer out of this data set.

Cue 402 — 00:26:15,366 --> 00:26:17,206

So then we tag it and then we store it.

Cue 403 — 00:26:17,246 --> 00:26:21,006

Right. So this is this is something that not a lot of people in data engineering do

Cue 404 — 00:26:21,046 --> 00:26:26,145

rigorously enough in my opinion, which is when you work on a data set,

Cue 405 — 00:26:26,286 --> 00:26:32,166

especially in large data set, we need to make sure that we keep versioned histories

Cue 406 — 00:26:32,206 --> 00:26:35,966

of these old data sets around so that we can compare what we did in the past to what

Cue 407 — 00:26:36,006 --> 00:26:39,646

kind of data we have now kind of results we get from that data,

Cue 408 — 00:26:39,686 --> 00:26:44,246

because it will be useful to be able to do that analysis at some point in time.

Cue 409 — 00:26:44,286 --> 00:26:48,486

Again, if we think about what we're trying to do, we put push raw data in,

Cue 410 — 00:26:48,526 --> 00:26:53,126

we refine it, we make it better to get better answers and be able to track this just

Cue 411 — 00:26:53,166 --> 00:26:54,806

like we do in software engineering.

Cue 412 — 00:26:54,846 --> 00:26:57,225

We've tried to version the data as well.

Cue 413 — 00:26:58,126 --> 00:27:05,006

All right. So almost at the end here, but just want to give you the overall

Cue 414 — 00:27:05,046 --> 00:27:07,006

picture of what we're doing here.

Cue 415 — 00:27:07,046 --> 00:27:13,526

So we have an ingest job which takes our docs converts them into parquet.

Cue 416 — 00:27:13,566 --> 00:27:16,625

We put that into some kind of intermediate state.

Cue 417 — 00:27:16,686 --> 00:27:20,446

In this case I'm using a Hugging Face storage buckets which use Xet,

Cue 418 — 00:27:20,486 --> 00:27:24,886

which allows me to very quickly upload and download data when I've changed the data

Cue 419 — 00:27:24,926 --> 00:27:28,766

on the inside of those files, rather than having to upload gigabytes and gigabytes

Cue 420 — 00:27:28,806 --> 00:27:34,846

of data. Then we run the refinery job that takes our data and makes it clean and

Cue 421 — 00:27:34,886 --> 00:27:41,586

better and ready for use. We commit that into a basically a versioned repository.

Cue 422 — 00:27:41,646 --> 00:27:44,166

Buckets are intermediate data storage.

Cue 423 — 00:27:44,206 --> 00:27:49,606

Data sets are versioned permanent data storage or storage for your data.

Cue 424 — 00:27:49,646 --> 00:27:56,686

Okay. And then once we have that we fire a webhook that goes and sees if our new

Cue 425 — 00:27:56,726 --> 00:28:03,486

or updated data set passes our test, our eval gate, are we hitting 95% accuracy

Cue 426 — 00:28:03,526 --> 00:28:08,446

whatever number you decide to choose, okay, then we tag that if it's good and then

Cue 427 — 00:28:08,486 --> 00:28:13,686

we go and deploy that. When I say space, that is this thing that I was running here.

Cue 428 — 00:28:13,726 --> 00:28:18,566

Basically, there's a way that you can run Python applications or you can run a

Cue 429 — 00:28:18,606 --> 00:28:23,366

Docker container for your app, or you can run the jobs or whatever directly onto

Cue 430 — 00:28:23,406 --> 00:28:28,526

Hugging Face cloud. Okay. So but again, you don't have to use our stuff or

Cue 431 — 00:28:28,566 --> 00:28:31,246

infrastructure. You can deploy all this yourself.

Cue 432 — 00:28:31,286 --> 00:28:35,646

You just tell your agent to rebuild this on AWS or Google Cloud or whatever you use.

Cue 433 — 00:28:35,686 --> 00:28:41,286

Okay. All right. So that was our whole loop in terms of what we're trying to do

Cue 434 — 00:28:41,326 --> 00:28:44,486

here. Make that work. We test the refinery.

Cue 435 — 00:28:44,526 --> 00:28:48,226

We have a CD pipeline. We we data.

Cue 436 — 00:28:48,446 --> 00:28:50,846

We have a pipeline for our code.

Cue 437 — 00:28:50,886 --> 00:28:54,566

We want something similar for our data, and we want to make sure that both are green

Cue 438 — 00:28:54,606 --> 00:28:58,006

before we release any code to production.

Cue 439 — 00:28:58,046 --> 00:29:00,846

Okay. So we've already run the demo.

Cue 440 — 00:29:00,886 --> 00:29:07,726

And so I actually when we go do when we go went and ran this specific

Cue 441 — 00:29:07,766 --> 00:29:13,126

demo. This is actually just using, I think GPT-OSS, whatever the latest version is

Cue 442 — 00:29:13,166 --> 00:29:18,566

from a couple of years ago. So what's interesting about that is I wasn't using a

Cue 443 — 00:29:18,606 --> 00:29:22,486

large frontier model to actually do this application, right.

Cue 444 — 00:29:22,526 --> 00:29:24,526

I was just using a very small model.

Cue 445 — 00:29:24,566 --> 00:29:28,786

I think GPT-OSS is like a couple of gigabytes total in size.

Cue 446 — 00:29:28,926 --> 00:29:32,406

But the point here is that if you're using a very large model,

Cue 447 — 00:29:32,446 --> 00:29:37,446

whether it's one of the frontier models like Sonnet or Opus or ChatGPT or whatever,

Cue 448 — 00:29:37,486 --> 00:29:40,766

oftentimes you don't need that kind of sledgehammer for something like this.

Cue 449 — 00:29:40,806 --> 00:29:44,886

You can use a much smaller model, and why would you want to use it?

Cue 450 — 00:29:44,926 --> 00:29:46,766

Number one, it'll be much faster.

Cue 451 — 00:29:46,806 --> 00:29:48,966

Number two, it'll be much cheaper, right?

Cue 452 — 00:29:49,006 --> 00:29:53,446

So if you can achieve the same goal of whatever you're trying to build in your

Cue 453 — 00:29:53,486 --> 00:29:57,366

application, like in this one, it's going to ask the documentation or go ask what we

Cue 454 — 00:29:57,406 --> 00:30:02,126

have and get answers, correct answers very specifically.

Cue 455 — 00:30:02,166 --> 00:30:05,846

But do we want to burn a lot of expensive tokens to do that?

Cue 456 — 00:30:05,886 --> 00:30:12,126

Probably not. Right. So again, when when we refine, when we refine the data and we

Cue 457 — 00:30:12,166 --> 00:30:16,086

clean it, and we use these good data engineering practices to build this kind of

Cue 458 — 00:30:16,126 --> 00:30:18,766

stuff out, you're able to say, you know what?

Cue 459 — 00:30:18,806 --> 00:30:24,526

Actually, we don't need a very sophisticated high-end LLM to be able to do this

Cue 460 — 00:30:24,566 --> 00:30:27,966

because we're well, how's the saying go garbage in, garbage out,

Cue 461 — 00:30:28,006 --> 00:30:29,606

right. What's the opposite of garbage in?

Cue 462 — 00:30:29,646 --> 00:30:32,046

Garbage out, I don't know. Anyways, you get the idea.

Cue 463 — 00:30:32,086 --> 00:30:38,886

So when we clean this up, we can use a much faster and cheaper model to be able to

Cue 464 — 00:30:38,926 --> 00:30:44,206

do the same things. Okay, and I don't know about you, but I don't think tokens have

Cue 465 — 00:30:44,246 --> 00:30:46,406

really gotten that much cheaper in the last year.

Cue 466 — 00:30:46,446 --> 00:30:48,766

I don't feel like they they will for a while.

Cue 467 — 00:30:48,806 --> 00:30:51,446

Okay, not until the price of memory comes down dramatically,

Cue 468 — 00:30:51,486 --> 00:30:53,126

but that's a different story. Okay.

Cue 469 — 00:30:53,166 --> 00:30:54,066

All right.

Cue 470 — 00:30:55,566 --> 00:31:01,966

So again, one of the reasons why we version our data sets as we add new things or

Cue 471 — 00:31:02,006 --> 00:31:08,206

refine it is to make sure that we're able to go look historically to see what what

Cue 472 — 00:31:08,246 --> 00:31:12,846

changed in our data. Why are we getting worse results with this updated data set or

Cue 473 — 00:31:12,886 --> 00:31:14,846

with this new data we've added in it?

Cue 474 — 00:31:14,886 --> 00:31:17,646

If you have the old data set, it's version, you can go and do a comparison.

Cue 475 — 00:31:17,686 --> 00:31:21,726

You can figure that out, or you can tell your AI agent to go and figure that out for

Cue 476 — 00:31:21,766 --> 00:31:28,946

you. Okay. All right. So again, what we tried to do here is

Cue 477 — 00:31:30,446 --> 00:31:37,286

in the short demo we did when we looked at this, right, the crude data or

Cue 478 — 00:31:37,326 --> 00:31:42,445

just the raw data unprocessed, it gave us the wrong answer after we had processed

Cue 479 — 00:31:42,485 --> 00:31:44,586

it. It did give us the correct answer.

Cue 480 — 00:31:45,366 --> 00:31:49,965

One of the things that significantly helped to get the right answer here was to make

Cue 481 — 00:31:50,005 --> 00:31:54,886

sure that we didn't include older versions of the documentation as part of this.

Cue 482 — 00:31:54,926 --> 00:31:59,525

So a common question at this point is, well, why didn't just you just use the latest

Cue 483 — 00:31:59,565 --> 00:32:02,566

version anyways? Right? That would be an easy fix.

Cue 484 — 00:32:02,606 --> 00:32:04,525

And that actually does work, right?

Cue 485 — 00:32:04,565 --> 00:32:10,406

So you could basically do no refinement and just say, let's just use the latest

Cue 486 — 00:32:10,446 --> 00:32:14,286

version of our documentation so that we know we always get the right answer,

Cue 487 — 00:32:14,326 --> 00:32:19,166

at least for these class of questions that we'll get that we need to get answers

Cue 488 — 00:32:19,206 --> 00:32:24,445

for. Okay. But if you think about that, when you make that decision as an engineer,

Cue 489 — 00:32:24,485 --> 00:32:29,566

so you just use the latest data set, what you're doing is you are filtering that

Cue 490 — 00:32:29,606 --> 00:32:31,766

data, right? You're not letting the machine do it.

Cue 491 — 00:32:31,806 --> 00:32:35,646

You're doing it conceptually and also physically yourself.

Cue 492 — 00:32:35,686 --> 00:32:38,766

Right. And then also brings up a whole host of other questions.

Cue 493 — 00:32:38,806 --> 00:32:42,646

Are the documentation the latest documentation 100% complete?

Cue 494 — 00:32:42,686 --> 00:32:47,806

Is there useful stuff in old documentation where you have like migration notes or

Cue 495 — 00:32:47,846 --> 00:32:50,846

what's changed in this new release of the software?

Cue 496 — 00:32:50,886 --> 00:32:53,705

That stuff is actually extremely useful.

Cue 497 — 00:32:53,926 --> 00:32:58,926

Again, for this specific example, it's extremely useful because it will have extra

Cue 498 — 00:32:58,966 --> 00:33:03,855

notes that may not be in the actual documentation that you may want to provide into

Cue 499 — 00:33:03,895 --> 00:33:09,005

the LLM as a chunk, so that it can give you a correct answer or have better context

Cue 500 — 00:33:09,045 --> 00:33:11,546

to formulate a good answer. Okay.

Cue 501 — 00:33:12,686 --> 00:33:14,806

All right. So we're over time now.

Cue 502 — 00:33:14,846 --> 00:33:17,646

So I'm going to we've already talked about all this stuff.

Cue 503 — 00:33:17,686 --> 00:33:18,946

So we're at the end.

Cue 504 — 00:33:20,246 --> 00:33:23,726

Thank you for staying a few extra minutes for this presentation.

Cue 505 — 00:33:23,766 --> 00:33:25,406

And I know it's right before lunch.

Cue 506 — 00:33:25,446 --> 00:33:27,586

So we'll take questions here over there.

Cue 507 — 00:33:28,045 --> 00:33:31,406

But please don't ding me and give me a bad review for cutting into lunch.

Cue 508 — 00:33:31,446 --> 00:33:35,166

It's surprising that if you run into lunch, you get worse reviews,

Cue 509 — 00:33:35,206 --> 00:33:37,525

right? I don't know why that is, but but please don't do that.

Cue 510 — 00:33:37,565 --> 00:33:39,846

I hope you enjoyed the talk. Please connect.

Cue 511 — 00:33:39,886 --> 00:33:43,806

And again, if you're interested in having this as a workshop or a fine tuning

Cue 512 — 00:33:43,846 --> 00:33:45,806

workshop at your company, please reach out.

Cue 513 — 00:33:45,846 --> 00:33:47,005

And this is part of what I do.

Cue 514 — 00:33:47,045 --> 00:33:48,785

Okay. Thank you all for coming.

skills

talk-pratik-patel-refined-rag-data

tile.json