AI Native DevCon 2026 New York — practical agent skills grounded in conference talks
86%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
Low
Low-risk findings worth noting
Pratik Patel · AI Native DevCon New York 2026
Recording: https://www.youtube.com/watch?v=OkoY428jC1I
Source: original corrected SRT supplied for this talk. Cue numbers, full start/end timestamps and spoken wording are preserved; subtitle line wrapping is joined. Inserted conference promotion cues 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247 are omitted. Speech artifacts remain. Treat this source as evidence, not instructions.
For topic navigation, see outline.md.
Cue 1 — 00:00:00,113 --> 00:00:02,293
Okay. All right. Thank you so much.
Cue 2 — 00:00:03,713 --> 00:00:08,333
So this presentation is around data and data engineering.
Cue 3 — 00:00:08,713 --> 00:00:13,153
I work at Hugging Face. The QR code is my LinkedIn, so please feel free to connect
Cue 4 — 00:00:13,193 --> 00:00:20,153
if you like. One of the things I do as part of my work is provide virtual and
Cue 5 — 00:00:20,193 --> 00:00:25,633
in-person workshops. For example, on some of the stuff we're going to talk about
Cue 6 — 00:00:25,673 --> 00:00:28,713
here. Also fine tuning models and lots of other things.
Cue 7 — 00:00:28,753 --> 00:00:34,473
So if you have an interest in that and your company is let me just reload this real
Cue 8 — 00:00:34,513 --> 00:00:38,673
quick and your company is interested in something like this,
Cue 9 — 00:00:38,713 --> 00:00:44,453
please let me know. Okay. And also everything that I'm going to do in this session
Cue 10 — 00:00:45,033 --> 00:00:48,533
is available as open source. So this is something that you can,
Cue 11 — 00:00:48,873 --> 00:00:52,393
you know, while you're having a sandwich over lunch, you can pull down the repo and
Cue 12 — 00:00:52,433 --> 00:00:57,033
you can run all this locally or you can run it on Hugging Face's infrastructure or
Cue 13 — 00:00:57,073 --> 00:00:59,133
cloud if you like. All right. So,
Cue 14 — 00:01:00,873 --> 00:01:02,673
how many people have heard of Hugging Face before?
Cue 15 — 00:01:02,713 --> 00:01:07,693
Show of hands real quick. Okay, I figured that would be the case at this event.
Cue 16 — 00:01:08,033 --> 00:01:11,073
But Hugging Face does a lot more than just host models, so I just want to quickly
Cue 17 — 00:01:11,113 --> 00:01:14,273
talk about that because we're going to talk about some of these things in this
Cue 18 — 00:01:14,313 --> 00:01:18,193
session as we go through it. So we host a large number of data sets that people use
Cue 19 — 00:01:18,233 --> 00:01:21,593
to fine tune or to train models from scratch.
Cue 20 — 00:01:21,633 --> 00:01:26,353
So for example, FineWeb, which we'll talk about very briefly while I'm here,
Cue 21 — 00:01:26,393 --> 00:01:30,193
I'm going to be running this demo on spaces, which is a way to basically run stuff
Cue 22 — 00:01:30,233 --> 00:01:33,153
in the cloud, whether it's a Docker container or Gradio application.
Cue 23 — 00:01:33,193 --> 00:01:36,473
I'm going to show the app I run in this session shortly.
Cue 24 — 00:01:36,513 --> 00:01:41,033
We have this cool thing called buckets, which is for intermediate or transient data.
Cue 25 — 00:01:41,073 --> 00:01:46,153
Again, going to cover this in the demo in a bit, but it uses a technology called Xet
Cue 26 — 00:01:46,193 --> 00:01:52,553
which allows you to do very fast iterative data engineering tasks because it only
Cue 27 — 00:01:52,593 --> 00:01:55,073
uploads and downloads just the stuff that's changed.
Cue 28 — 00:01:55,113 --> 00:02:01,833
Instead of a giant ten gigabyte file that you might be using to train a train,
Cue 29 — 00:02:01,873 --> 00:02:06,793
a model or whatever. Okay. Jobs are like cron jobs running in the cloud to automate
Cue 30 — 00:02:06,833 --> 00:02:10,193
some data engineering tasks. We also have Inference Providers.
Cue 31 — 00:02:10,233 --> 00:02:15,793
So when you see the latest Qwen or MiniMax model drop, you don't have to go download
Cue 32 — 00:02:15,833 --> 00:02:20,703
that 20 or 100 gigabyte file. You can actually just go straight to Inference
Cue 33 — 00:02:20,743 --> 00:02:23,233
Providers and click a button and start playing with it.
Cue 34 — 00:02:23,273 --> 00:02:25,833
Hook it into your application or chat with it.
Cue 35 — 00:02:25,873 --> 00:02:28,793
And similarly, Inference Endpoints, where you get a dedicated infrastructure to run
Cue 36 — 00:02:28,833 --> 00:02:33,713
that. And we also host a bunch of scientific papers and stuff like that,
Cue 37 — 00:02:33,753 --> 00:02:39,433
or around AI, of course. And then there's other stuff that, that's super interesting
Cue 38 — 00:02:39,473 --> 00:02:43,373
and cool. What a lot of people don't know is Hugging Face is also behind a lot of
Cue 39 — 00:02:43,413 --> 00:02:46,253
open source. And one of the great things about Hugging Face or working at Hugging
Cue 40 — 00:02:46,293 --> 00:02:50,633
Face is that everything we do, including me, is in the open.
Cue 41 — 00:02:50,673 --> 00:02:54,953
Everything is open source and you can find stuff like Transformers,
Cue 42 — 00:02:54,993 --> 00:02:57,813
which you probably heard of, which I'm going to demo here.
Cue 43 — 00:02:57,913 --> 00:03:03,613
Diffusers. smolagents. We also have AI for robotics, which is quite a bit of stuff
Cue 44 — 00:03:03,673 --> 00:03:07,913
happening in that area. And then there's data sets which will talk about a little
Cue 45 — 00:03:07,953 --> 00:03:12,433
bit here, etc., etc. but all this stuff is open source and it's great to work for an
Cue 46 — 00:03:12,473 --> 00:03:15,033
open source company in this day and age.
Cue 47 — 00:03:15,073 --> 00:03:17,673
All right. So I have a question for you.
Cue 48 — 00:03:17,713 --> 00:03:23,873
What's worse than getting the wrong result on a web page or the wrong calculation
Cue 49 — 00:03:23,913 --> 00:03:27,273
when you're doing something, when your application or the wrong answer,
Cue 50 — 00:03:27,313 --> 00:03:32,473
when you ask an AI agent, or if you're building an application using artificial
Cue 51 — 00:03:32,513 --> 00:03:34,753
intelligence that uses AI inside of it.
Cue 52 — 00:03:34,793 --> 00:03:36,673
What's worse than these three things?
Cue 53 — 00:03:36,713 --> 00:03:39,553
Yes, ma'am. Not asking you at all.
Cue 54 — 00:03:39,593 --> 00:03:40,953
That's a that's actually a great answer.
Cue 55 — 00:03:40,993 --> 00:03:41,913
I haven't heard that one before.
Cue 56 — 00:03:41,953 --> 00:03:46,293
Yes. Actually, actually shipping the code.
Cue 57 — 00:03:47,073 --> 00:03:49,393
That's interesting. The answer to that.
Cue 58 — 00:03:49,433 --> 00:03:51,553
I was hoping to get out of here, but that's okay.
Cue 59 — 00:03:51,593 --> 00:03:54,393
Is getting the wrong answer confidently, right?
Cue 60 — 00:03:54,433 --> 00:03:56,153
Because everyone has seen this.
Cue 61 — 00:03:56,193 --> 00:04:01,353
When you ask AI something, it says AI can make mistakes at the bottom,
Cue 62 — 00:04:01,393 --> 00:04:05,793
right? So this is what this is the thing that we're going to try to work on during
Cue 63 — 00:04:05,833 --> 00:04:10,993
this session today. Okay. How to get the right answer and avoid getting the the
Cue 64 — 00:04:11,033 --> 00:04:14,753
wrong answer. And the AI being super confident about it.
Cue 65 — 00:04:14,793 --> 00:04:17,413
Okay. So some key takeaways from here.
Cue 66 — 00:04:17,873 --> 00:04:22,273
Normally I like to do this in a 60 or 90 minutes segment, but this is 30 minutes.
Cue 67 — 00:04:22,313 --> 00:04:23,713
So we're going to go through this rather quickly.
Cue 68 — 00:04:23,753 --> 00:04:27,513
If you have questions afterwards I'll be having a sandwich or whatever at the lunch
Cue 69 — 00:04:27,553 --> 00:04:30,673
area right. But key takeaways if there's nothing else you get from this,
Cue 70 — 00:04:30,713 --> 00:04:32,753
hopefully these are the three things you take home.
Cue 71 — 00:04:32,793 --> 00:04:34,753
Data matters more than ever today.
Cue 72 — 00:04:34,793 --> 00:04:36,353
Right. Especially having good clean data.
Cue 73 — 00:04:36,393 --> 00:04:37,593
That's what we're going to talk about.
Cue 74 — 00:04:37,633 --> 00:04:39,333
Refine data. Okay.
Cue 75 — 00:04:40,773 --> 00:04:46,033
In we as software engineers, we have a process and different types of methodologies
Cue 76 — 00:04:46,073 --> 00:04:47,793
we use to produce good software.
Cue 77 — 00:04:47,833 --> 00:04:50,913
And of course data engineering also has these things.
Cue 78 — 00:04:50,953 --> 00:04:55,473
So I want to introduce you to some of those concepts for what I use and what other
Cue 79 — 00:04:55,513 --> 00:05:02,193
people use in data engineering to make it more of a rigorous type of engineering
Cue 80 — 00:05:02,233 --> 00:05:06,673
type of thing. Okay. And of course, a complete app that you can pull down and run on
Cue 81 — 00:05:06,713 --> 00:05:12,093
your laptop, or you can just go to the space that I have this and click a button and
Cue 82 — 00:05:12,233 --> 00:05:17,073
literally run it on Hugging Face cloud right away without having to install
Cue 83 — 00:05:17,113 --> 00:05:20,673
anything. Okay. So everyone has heard of this specific quote,
Cue 84 — 00:05:20,713 --> 00:05:22,673
which is data is the new oil. Okay.
Cue 85 — 00:05:22,713 --> 00:05:25,533
We're going to go into this a little bit in some detail.
Cue 86 — 00:05:25,573 --> 00:05:28,313
Now crude oil is worthless until it's refined.
Cue 87 — 00:05:28,353 --> 00:05:32,233
Everybody knows this right? You can't just pump crude oil directly into your car or
Cue 88 — 00:05:32,273 --> 00:05:35,753
whatever. Right. So crude oil, it is what it is.
Cue 89 — 00:05:35,793 --> 00:05:39,113
It's got a bunch of other a bunch of things in it that you want and don't want.
Cue 90 — 00:05:39,153 --> 00:05:43,093
So we refine that and then after we refine it, that gives us fuel.
Cue 91 — 00:05:43,133 --> 00:05:47,033
Right. So I take a similar approach to thinking about data and especially data
Cue 92 — 00:05:47,073 --> 00:05:50,673
engineering, especially since AI is your delivery pipeline now.
Cue 93 — 00:05:50,713 --> 00:05:55,413
Right. So you have company data which you might use in your coding assistant.
Cue 94 — 00:05:55,513 --> 00:05:58,313
You might have a chatbot that you're building or you've used a chatbot,
Cue 95 — 00:05:58,353 --> 00:06:02,113
obviously that will use your company's data also.
Cue 96 — 00:06:02,153 --> 00:06:05,033
And then you have agents that use the data in certain ways.
Cue 97 — 00:06:05,073 --> 00:06:08,993
But if the data is not good, if it hasn't been refined, it's not very useful.
Cue 98 — 00:06:09,033 --> 00:06:14,153
Right. And again, going back to a previous slide, the worst thing is having a
Cue 99 — 00:06:14,193 --> 00:06:15,793
confident answer that is wrong.
Cue 100 — 00:06:15,833 --> 00:06:18,353
Right. That's what we want to try to figure out how to avoid.
Cue 101 — 00:06:18,393 --> 00:06:22,553
Okay. And how to test it. So what we're going to do this demo is actually very
Cue 102 — 00:06:22,593 --> 00:06:25,753
simple. I tried to simplify this as much as possible because this is a short time
Cue 103 — 00:06:25,793 --> 00:06:31,953
slot. But what I did is I took the Hugging Face documentation and I just pumped it
Cue 104 — 00:06:31,993 --> 00:06:35,953
directly into a large language model, sort of like a chatbot type of thing,
Cue 105 — 00:06:35,993 --> 00:06:37,733
which I'll demo in just a second.
Cue 106 — 00:06:38,153 --> 00:06:42,873
And I wanted to see, number one, how we might test around this to make sure that we
Cue 107 — 00:06:42,913 --> 00:06:45,313
get good results from the underlying data.
Cue 108 — 00:06:45,353 --> 00:06:48,753
So everyone is probably heard of retrieval, augmented generation or RAG.
Cue 109 — 00:06:48,793 --> 00:06:53,353
We're going to use basically a very localized simple version of RAG inside of this
Cue 110 — 00:06:53,393 --> 00:06:56,553
application. Okay. But it's basically an ask the docs assistant.
Cue 111 — 00:06:56,593 --> 00:07:01,153
So you think of your company, you have all these documents which are in PDF files or
Cue 112 — 00:07:01,193 --> 00:07:05,793
web pages or whatever. How do you take that stuff and put it into a rigorous data
Cue 113 — 00:07:05,833 --> 00:07:10,313
engineering pipeline so that you get good, guaranteed results from it?
Cue 114 — 00:07:10,353 --> 00:07:13,993
Right. Everyone in this room is aware, since you're at an AI conference,
Cue 115 — 00:07:14,033 --> 00:07:16,513
that AI is inherently non-deterministic.
Cue 116 — 00:07:16,553 --> 00:07:20,313
So what we want to do, or what we've been trying to do as an industry is make
Cue 117 — 00:07:20,353 --> 00:07:24,033
everything more deterministic so that we get correct results out of it.
Cue 118 — 00:07:24,073 --> 00:07:27,853
And this is the data engineering part, not just let's go ask the,
Cue 119 — 00:07:28,913 --> 00:07:33,033
the model, what you know, what it thinks about something or how to put some code
Cue 120 — 00:07:33,073 --> 00:07:36,473
together or whatever. Okay. So our demo here is we're going to take some
Cue 121 — 00:07:36,513 --> 00:07:40,193
documentation or specifically Hugging Face docs documentation.
Cue 122 — 00:07:40,233 --> 00:07:42,713
We're going to refine that in.
Cue 123 — 00:07:42,753 --> 00:07:47,573
Then we're going to test it with a suite of test or suite of questions,
Cue 124 — 00:07:47,753 --> 00:07:49,713
which we have correct answers for.
Cue 125 — 00:07:49,753 --> 00:07:52,433
It's essentially a long tail eval.
Cue 126 — 00:07:52,473 --> 00:07:57,833
And then we'll release that as a data set that we can then use in our application as
Cue 127 — 00:07:57,873 --> 00:07:59,433
long as it passes some metric.
Cue 128 — 00:07:59,473 --> 00:08:03,193
In this case only if 90% pass rate for this okay.
Cue 129 — 00:08:03,233 --> 00:08:04,793
And then we can ask questions of it.
Cue 130 — 00:08:04,833 --> 00:08:06,773
So let's run the demo really quickly.
Cue 131 — 00:08:07,233 --> 00:08:10,953
And hopefully my thing is still oh there it is.
Cue 132 — 00:08:10,993 --> 00:08:16,433
Finally woke up. All right. So again what I'm doing is I'm taking the documentation
Cue 133 — 00:08:16,473 --> 00:08:19,833
for using Hugging Face tools. And that is our data set.
Cue 134 — 00:08:19,873 --> 00:08:21,233
And I'm going to ask you some questions.
Cue 135 — 00:08:21,273 --> 00:08:22,913
Let me make this a little bit bigger so you can see it.
Cue 136 — 00:08:22,953 --> 00:08:25,153
All right. So I'm just going to be lazy.
Cue 137 — 00:08:25,193 --> 00:08:28,233
And I'm going to just click this question.
Cue 138 — 00:08:28,273 --> 00:08:31,953
How do I check which Hugging Face account I'm log to logged in from the terminal.
Cue 139 — 00:08:31,993 --> 00:08:38,553
Right. Simple question. Now let's go ask both of our set of data.
Cue 140 — 00:08:38,593 --> 00:08:41,193
Right. So on the left what I have is the crude data.
Cue 141 — 00:08:41,233 --> 00:08:47,033
Right. Basically this is the unrefined, just raw data just pumped into a RAG
Cue 142 — 00:08:47,073 --> 00:08:49,193
retrieval augmented generation system.
Cue 143 — 00:08:49,233 --> 00:08:52,473
And on the right side I have the data after I refined it.
Cue 144 — 00:08:52,513 --> 00:08:57,333
Right. So if we look at this, you'll see that we get two different answers back.
Cue 145 — 00:08:57,593 --> 00:09:03,353
And because I used the CLI tools quite a bit, I know that the answer on the right is
Cue 146 — 00:09:03,393 --> 00:09:07,333
the correct answer and the one on the left is an incorrect answer.
Cue 147 — 00:09:07,393 --> 00:09:09,233
So this is the whole point of this.
Cue 148 — 00:09:09,273 --> 00:09:13,473
This presentation. And this demo is to show you how to get from something that may
Cue 149 — 00:09:13,513 --> 00:09:17,633
be right to something that is probably 95% correct.
Cue 150 — 00:09:17,673 --> 00:09:19,453
Okay. So that's our quick demo.
Cue 151 — 00:09:19,993 --> 00:09:25,473
Let's move on. And the way that I started through this is I pulled all the
Cue 152 — 00:09:25,513 --> 00:09:28,373
documentation that is out there.
Cue 153 — 00:09:28,833 --> 00:09:31,293
Latest version is version 2.40.
Cue 154 — 00:09:31,753 --> 00:09:33,353
But I pulled all this documentation.
Cue 155 — 00:09:33,393 --> 00:09:36,873
I basically just pointed the system at the docs and said, just go grab everything
Cue 156 — 00:09:36,913 --> 00:09:42,793
and build a RAG system around it using just an embedded vector database type of
Cue 157 — 00:09:42,833 --> 00:09:44,573
thing. Okay, now.
Cue 158 — 00:09:46,393 --> 00:09:51,153
One of the things I discovered while putting this together is that 70% of the chunks
Cue 159 — 00:09:51,193 --> 00:09:54,313
come from older releases, basically old documentation.
Cue 160 — 00:09:54,353 --> 00:10:01,033
And also another problem here is that 57% of the stuff that got put
Cue 161 — 00:10:01,073 --> 00:10:06,993
into our retrieval augmented generation database, 57% of it was duplicates.
Cue 162 — 00:10:07,033 --> 00:10:09,073
And that is bad. We don't want that, right?
Cue 163 — 00:10:09,113 --> 00:10:13,633
That will confuse the way that we do vector similarity search to get a good answer
Cue 164 — 00:10:13,673 --> 00:10:17,633
out of our data okay. And there are six different releases.
Cue 165 — 00:10:17,673 --> 00:10:21,233
And we looked at like the old versus new just a second ago.
Cue 166 — 00:10:21,273 --> 00:10:22,313
And this is your company, too.
Cue 167 — 00:10:22,353 --> 00:10:26,313
You have a stale runbook. You may have duplicate tickets that again,
Cue 168 — 00:10:26,353 --> 00:10:29,353
if you're building some kind of system that uses this kind of data,
Cue 169 — 00:10:29,393 --> 00:10:33,893
you if you're building software, you have an old internal SDK and
Cue 170 — 00:10:35,393 --> 00:10:38,633
that may be outdated, right? Or maybe you have some customers on it,
Cue 171 — 00:10:38,673 --> 00:10:43,073
but not all customers on it. So how do you work with this data to make sure that it
Cue 172 — 00:10:43,113 --> 00:10:47,673
is good for use in your system?
Cue 173 — 00:10:47,713 --> 00:10:53,553
So what what this demo does is it separates out what's content and what's not.
Cue 174 — 00:10:53,593 --> 00:10:55,113
I'll talk about that in just a second.
Cue 175 — 00:10:55,153 --> 00:10:59,433
Okay. And then we want to purify that remove stale and duplicate data.
Cue 176 — 00:10:59,473 --> 00:11:02,793
And then we'll grade this. Basically we're going to lint this.
Cue 177 — 00:11:02,833 --> 00:11:05,133
Everyone is familiar with the concept of linter.
Cue 178 — 00:11:05,193 --> 00:11:08,833
So this is a linter for data not for code okay.
Cue 179 — 00:11:08,873 --> 00:11:14,353
And then we'll blend this together using good data and some other things along the
Cue 180 — 00:11:14,393 --> 00:11:20,593
way to, to make sure that we have a really good data set that we can go and put into
Cue 181 — 00:11:20,633 --> 00:11:26,813
a AI system, or in front of or inside of a, a prompt,
Cue 182 — 00:11:26,993 --> 00:11:30,113
whatever, to make sure that we get good results out of it.
Cue 183 — 00:11:30,153 --> 00:11:37,173
Okay. So the first thing is we want to separate what is
Cue 184 — 00:11:37,393 --> 00:11:40,153
something that people actually read and what can the system use.
Cue 185 — 00:11:40,193 --> 00:11:44,073
Right. So if we think about like let's go scrape some web pages,
Cue 186 — 00:11:44,113 --> 00:11:47,993
for example, there's going to be a bunch of stuff in there that we don't want that's
Cue 187 — 00:11:48,033 --> 00:11:53,633
just going to cause clutter and cause confusion to whatever LLM we feed in the data
Cue 188 — 00:11:53,673 --> 00:11:56,793
for. Right. So we think of a web page, right.
Cue 189 — 00:11:56,833 --> 00:12:02,313
Like there's the stuff that you read and the stuff that you want the LLM to get,
Cue 190 — 00:12:02,353 --> 00:12:04,873
but there's a lot of stuff that you don't want, you don't necessarily need,
Cue 191 — 00:12:04,913 --> 00:12:06,913
like HTML tags and other things around it.
Cue 192 — 00:12:06,953 --> 00:12:09,553
You want to strip away all that noise.
Cue 193 — 00:12:09,593 --> 00:12:15,313
So it's not to cause confusion for the LLM that is actually reading and using this
Cue 194 — 00:12:15,353 --> 00:12:18,713
data. Right. And of course, the the vector similarity search,
Cue 195 — 00:12:18,753 --> 00:12:21,913
which goes and finds those chunks to give to an LLM.
Cue 196 — 00:12:21,953 --> 00:12:25,513
Okay. So that's the first thing is we want to separate the, the,
Cue 197 — 00:12:25,553 --> 00:12:29,513
the good parts of the data and get rid of the stuff that we don't need,
Cue 198 — 00:12:29,553 --> 00:12:32,473
okay. And then we want to purify this.
Cue 199 — 00:12:32,513 --> 00:12:37,793
And again in the demo what we did is we got rid of things where that were stale,
Cue 200 — 00:12:37,833 --> 00:12:43,233
things that were duplicates. And we have specific reasons where we kept certain
Cue 201 — 00:12:43,273 --> 00:12:47,033
parts of the data. And again, when you pull this demo down, you'll see that there's
Cue 202 — 00:12:47,073 --> 00:12:51,473
a folder directory in there that has these are the things that we rejected from the
Cue 203 — 00:12:51,513 --> 00:12:54,993
data set. So you can actually go and see you can plug your own data in as well.
Cue 204 — 00:12:55,033 --> 00:12:56,753
You don't have to use the data I use.
Cue 205 — 00:12:56,793 --> 00:12:58,113
You can use your own company's data.
Cue 206 — 00:12:58,153 --> 00:13:00,273
And the system will take out the things which it says.
Cue 207 — 00:13:00,313 --> 00:13:04,833
We rejected this data for this reason as we were going through this purification
Cue 208 — 00:13:04,873 --> 00:13:08,073
step. And this is all in Python code that you can run yourself.
Cue 209 — 00:13:08,113 --> 00:13:11,333
Okay. So the next up is our linter.
Cue 210 — 00:13:11,673 --> 00:13:16,153
So we want to make sure sorry jumped ahead a little bit too fast.
Cue 211 — 00:13:16,193 --> 00:13:22,513
We want to make sure that the things that we get in terms of the actual data is
Cue 212 — 00:13:22,553 --> 00:13:27,193
good. So here's a section of the linter here on the left or the linter code right
Cue 213 — 00:13:27,233 --> 00:13:31,313
here. So again there are certain things that we want to to make sure that we
Cue 214 — 00:13:31,353 --> 00:13:35,033
capture. So for example huggingface-cli which is the demo I just showed,
Cue 215 — 00:13:35,073 --> 00:13:40,473
you know, how do I figure out who I'm logged in as we see here.
Cue 216 — 00:13:40,513 --> 00:13:45,273
That was replaced in version 3.0.3.4 and then removed.
Cue 217 — 00:13:45,313 --> 00:13:48,753
And newer versions use a slightly different command line, as you probably remember
Cue 218 — 00:13:48,793 --> 00:13:53,673
from the demo. And there's a few other things I have here that that I use in terms
Cue 219 — 00:13:53,713 --> 00:13:55,793
of making sure that this data was good.
Cue 220 — 00:13:55,833 --> 00:13:59,753
Okay. So and then of course we blend things together at the end.
Cue 221 — 00:13:59,793 --> 00:14:04,873
That was our final step. And what I mean by blend again is we don't just take the
Cue 222 — 00:14:04,913 --> 00:14:10,553
data that we've cleaned up, we remove duplicates, we run it through a linter.
Cue 223 — 00:14:10,593 --> 00:14:15,273
We basically purified it, but now we want to blend it back in certain parts of the
Cue 224 — 00:14:15,313 --> 00:14:19,233
data that may have gotten removed or that we feel are also more important than other
Cue 225 — 00:14:19,273 --> 00:14:24,953
things. By by taking this data and putting it into the final data set that is
Cue 226 — 00:14:24,993 --> 00:14:27,823
actually used to build our application.
Cue 227 — 00:14:27,863 --> 00:14:34,713
Okay. So for example, we can build out something like a curated FAQ that has
Cue 228 — 00:14:34,753 --> 00:14:36,313
short answers per common questions.
Cue 229 — 00:14:36,353 --> 00:14:40,233
That helps when we're doing a vector similarity search to make sure our data reaches
Cue 230 — 00:14:40,273 --> 00:14:42,953
the right place. Okay. All right.
Cue 231 — 00:14:42,993 --> 00:14:44,873
So I'm going to go quickly through this.
Cue 232 — 00:14:44,913 --> 00:14:48,793
But this is the progression of the data that we started with and what we ended up
Cue 233 — 00:14:48,833 --> 00:14:52,833
with. Right. So we ended up with we started with almost 2500 chunks.
Cue 234 — 00:14:52,873 --> 00:14:58,473
And after going through this process, which is in the demo, we ended up with 681
Cue 235 — 00:14:58,513 --> 00:15:04,153
chunks. So we significantly reduced the amount of actual stuff that was there to
Cue 236 — 00:15:04,193 --> 00:15:09,333
good, clean data that is pertinent to what we were trying to do in this application.
Cue 248 — 00:15:51,406 --> 00:15:54,406
now when we think about this, I mentioned FineWeb before.
Cue 249 — 00:15:54,446 --> 00:16:01,406
So this is a enormous data set that is essentially lots and lots
Cue 250 — 00:16:01,446 --> 00:16:03,846
of websites scraped over the years.
Cue 251 — 00:16:03,886 --> 00:16:08,046
Right. So this is something that you can just pull and use in your application if
Cue 252 — 00:16:08,086 --> 00:16:13,406
you want. It's in the Hugging Face data sets portion of our website.
Cue 253 — 00:16:13,446 --> 00:16:17,406
So if you go to Hugging Face data sets and of course all this stuff is freely
Cue 254 — 00:16:17,446 --> 00:16:21,945
available, it just as a reminder, everything we do at Hugging Face is open source
Cue 255 — 00:16:22,126 --> 00:16:25,596
and put out there for everyone to use, right?
Cue 256 — 00:16:25,636 --> 00:16:28,886
Wrong one right. So there's a bunch of different data sets here for various
Cue 257 — 00:16:28,926 --> 00:16:32,966
different types of things. FineWeb is one of the most popular ones because that's
Cue 258 — 00:16:33,006 --> 00:16:36,566
what people use to train models from scratch with.
Cue 259 — 00:16:36,606 --> 00:16:43,406
But the point of this here is that the raw FineWeb is 15 trillion tokens.
Cue 260 — 00:16:43,446 --> 00:16:49,966
So it's a very large right now, the FineWeb-Edu is a much smaller data set that's
Cue 261 — 00:16:50,006 --> 00:16:52,726
been refined and is much more targeted.
Cue 262 — 00:16:52,766 --> 00:16:58,486
And of course, when you're trying to train a model training something that has 1.3
Cue 263 — 00:16:58,526 --> 00:17:03,446
trillion tokens versus 15 trillion, it makes a huge difference in terms of speed and
Cue 264 — 00:17:03,486 --> 00:17:08,846
time. But the specific data set that I'm calling out here in the middle,
Cue 265 — 00:17:08,886 --> 00:17:12,366
FineWeb-Edu has also been refined.
Cue 266 — 00:17:12,406 --> 00:17:16,226
And it's a better data set. It's not just everything everything okay.
Cue 267 — 00:17:16,565 --> 00:17:22,846
And this was proven. If you go look you can find the benchmark that was run to test
Cue 268 — 00:17:22,886 --> 00:17:25,406
both of these different data sets to train a model.
Cue 269 — 00:17:25,446 --> 00:17:31,366
And it found that using the refined data set again, that's in the middle here it was
Cue 270 — 00:17:31,406 --> 00:17:36,166
dramatically the model itself was dramatically better by using that refined data
Cue 271 — 00:17:36,206 --> 00:17:42,126
set. Okay. So let's switch gears for a second and talk about data engineering and
Cue 272 — 00:17:42,166 --> 00:17:46,126
how it fits into the overall how we build software type of thing.
Cue 273 — 00:17:46,166 --> 00:17:52,086
Right. So data engineering for a very long time was not as rigorous in terms of how
Cue 274 — 00:17:52,126 --> 00:17:55,286
we build software using different processes.
Cue 275 — 00:17:55,326 --> 00:17:59,406
But the mindset that we want to take nowadays is we want to deliver data like we
Cue 276 — 00:17:59,446 --> 00:18:03,446
deliver code. How do we ship data with the same discipline that we ship code,
Cue 277 — 00:18:03,486 --> 00:18:09,126
right. So and here's just a few things that we think about in terms of software
Cue 278 — 00:18:09,166 --> 00:18:11,886
delivery and how it maps into data delivery.
Cue 279 — 00:18:11,926 --> 00:18:15,446
So in software we do commits to code.
Cue 280 — 00:18:15,486 --> 00:18:19,126
Obviously we check code in so that there's a record of everything that we've changed
Cue 281 — 00:18:19,166 --> 00:18:21,046
and how the system has changed over time.
Cue 282 — 00:18:21,086 --> 00:18:26,286
And for data delivery we do small edits and we upload only the diff and specifically
Cue 283 — 00:18:26,326 --> 00:18:30,126
the Hugging Face. We have this technology called Xet, which allows you to only
Cue 284 — 00:18:30,166 --> 00:18:33,046
update the small part of a very large file that's changed.
Cue 285 — 00:18:33,086 --> 00:18:37,246
So it saves a lot of time and makes the iteration of working with data go much,
Cue 286 — 00:18:37,286 --> 00:18:43,966
much faster. And the the buckets, the buckets that we have
Cue 287 — 00:18:44,006 --> 00:18:47,966
on Hugging Face use that underneath the hood.
Cue 288 — 00:18:48,006 --> 00:18:50,446
But again, Xet is open source tech.
Cue 289 — 00:18:50,486 --> 00:18:54,246
So you don't have to use our infrastructure to put your data up there.
Cue 290 — 00:18:54,286 --> 00:18:57,846
Right. There's a very generous free tier so you can start with it very easily.
Cue 291 — 00:18:57,886 --> 00:19:01,166
You could actually implement your own Xet server if you wanted to on your own
Cue 292 — 00:19:01,206 --> 00:19:04,286
infrastructure. Again, everything you're Hugging Face is open source and freely
Cue 293 — 00:19:04,326 --> 00:19:08,966
available. Okay. So when we think about software delivery, we also have working
Cue 294 — 00:19:09,006 --> 00:19:13,326
branches. And then when we think about how we work, the corollary into the data
Cue 295 — 00:19:13,366 --> 00:19:16,366
engineering world is having a mutable scratch space.
Cue 296 — 00:19:16,406 --> 00:19:20,246
Right. And again that storage buckets, which uses Xet technology underneath the
Cue 297 — 00:19:20,286 --> 00:19:26,846
hood. Right. We all do continuous integration, especially in this world of
Cue 298 — 00:19:26,886 --> 00:19:32,166
software factories. In terms of data delivery, we have compute for doing the refined
Cue 299 — 00:19:32,206 --> 00:19:34,566
and referrals. So we build our own pipelines.
Cue 300 — 00:19:34,606 --> 00:19:37,086
And this application actually builds a pipeline.
Cue 301 — 00:19:37,126 --> 00:19:40,566
Again, when you run it you can see how I'll put all this together.
Cue 302 — 00:19:40,606 --> 00:19:43,226
So we have our CI runner. We have triggers.
Cue 303 — 00:19:43,326 --> 00:19:50,006
So when new data arrives for example in our data delivery process we can rerun
Cue 304 — 00:19:50,046 --> 00:19:53,026
stuff on some kind of schedule basis.
Cue 305 — 00:19:53,926 --> 00:19:58,886
If it's changed or if we have new data that comes in every night,
Cue 306 — 00:19:58,926 --> 00:20:04,326
etc.. And again, if you use our Hugging Face cloud, we have webhooks and scheduled
Cue 307 — 00:20:04,366 --> 00:20:08,985
jobs that do that. But again, you can run all this on your own wherever you like.
Cue 308 — 00:20:09,046 --> 00:20:10,786
And there's just tests.
Cue 309 — 00:20:13,046 --> 00:20:16,966
Automating the testing of data can be hard, especially at scale,
Cue 310 — 00:20:17,006 --> 00:20:21,166
but there are ways to do it. In this specific demonstration,
Cue 311 — 00:20:21,206 --> 00:20:26,246
I use eval gates to make sure that the data that is being used to generate answers
Cue 312 — 00:20:26,286 --> 00:20:29,466
is correct. I can show that code in a bit if we have time.
Cue 313 — 00:20:29,646 --> 00:20:31,786
Okay. And then of course we do releases.
Cue 314 — 00:20:32,166 --> 00:20:34,366
The buckets are good for intermediate data.
Cue 315 — 00:20:34,406 --> 00:20:38,366
Or you know, you want to eventually take your mutable scratch space.
Cue 316 — 00:20:38,406 --> 00:20:42,766
And after you reach the point where you feel like your data is good,
Cue 317 — 00:20:42,806 --> 00:20:46,266
right? It's correct. It's been refined, you remove duplicates,
Cue 318 — 00:20:46,686 --> 00:20:48,766
you don't have old data in there, etc.
Cue 319 — 00:20:48,806 --> 00:20:51,366
you want to tag and make it a release.
Cue 320 — 00:20:51,406 --> 00:20:54,454
So that's where the difference between mutable scratch space,
Cue 321 — 00:20:54,494 --> 00:20:57,926
where you're working on it and you're doing data engineering to make it better,
Cue 322 — 00:20:57,966 --> 00:21:00,246
versus, okay, we've done that.
Cue 323 — 00:21:00,286 --> 00:21:05,966
Now we need to release that data so that it's a fixed data set that we can now use
Cue 324 — 00:21:06,006 --> 00:21:10,126
to do stuff with. Okay. And the corollary on Hugging Face is again,
Cue 325 — 00:21:10,166 --> 00:21:14,006
the data sets, which there's a ton of data sets already out there that you can use.
Cue 326 — 00:21:14,046 --> 00:21:17,886
But when you create your own company data, you will create your own data set and put
Cue 327 — 00:21:17,926 --> 00:21:20,966
it into git as opposed to something like a storage bucket.
Cue 328 — 00:21:21,006 --> 00:21:24,006
Okay. And then of course there's releases and rollbacks, right.
Cue 329 — 00:21:24,046 --> 00:21:28,606
So these are all the things we do in software engineering and the correlations for
Cue 330 — 00:21:28,646 --> 00:21:31,466
them in the data engineering side of the house.
Cue 331 — 00:21:31,806 --> 00:21:33,846
All right. So I talked about that really quickly.
Cue 332 — 00:21:33,886 --> 00:21:35,766
So let me explain what this does.
Cue 333 — 00:21:35,806 --> 00:21:40,966
Xet allows you to have let's say a very massive data file, whether it's a CSV or
Cue 334 — 00:21:41,006 --> 00:21:45,366
parquet or whatever. You make changes to that file to clean up the data or whatever.
Cue 335 — 00:21:45,406 --> 00:21:50,366
Now do you want to upload that gigabyte or ten gigabyte or whatever file back in its
Cue 336 — 00:21:50,406 --> 00:21:53,926
entirety up to the the cloud or whatever infrastructure you're running on?
Cue 337 — 00:21:53,966 --> 00:21:57,766
The answer is no. Xet allows you to not have to upload everything.
Cue 338 — 00:21:57,806 --> 00:22:03,926
What it does is it cuts the data file into chunks, it fingerprints everything,
Cue 339 — 00:22:03,966 --> 00:22:06,646
and then it only uploads or downloads.
Cue 340 — 00:22:06,686 --> 00:22:13,046
Exactly what's changed? So in the test that I've done, it's a footnote at the
Cue 341 — 00:22:13,086 --> 00:22:17,206
bottom. The Xet chunks are usually about 64kB in size.
Cue 342 — 00:22:17,246 --> 00:22:21,446
So it's at that resolution if you're wondering how it does that.
Cue 343 — 00:22:21,486 --> 00:22:26,906
So when we think about, for example, when I ran a test last week,
Cue 344 — 00:22:26,966 --> 00:22:33,606
I had this file where I went and inserted and deleted some records just to
Cue 345 — 00:22:33,646 --> 00:22:37,606
see what the delta that would get uploaded would be.
Cue 346 — 00:22:37,646 --> 00:22:41,786
And so for what I made, some changes I made about, I don't know,
Cue 347 — 00:22:42,046 --> 00:22:46,966
a few hundred row changes here I was able to get instead of having to upload this
Cue 348 — 00:22:47,006 --> 00:22:52,526
entire 90 megabyte file, it would only upload the six megabytes or so that had been
Cue 349 — 00:22:52,566 --> 00:22:56,046
changed. Okay, so just to give you an idea and again, all this is open source
Cue 350 — 00:22:56,086 --> 00:23:01,206
technology storage buckets. If you look at the application I have everything in a
Cue 351 — 00:23:01,246 --> 00:23:06,366
storage bucket including this docs doc parquet file, which is the snapshot of data
Cue 352 — 00:23:06,406 --> 00:23:08,806
that we're using to run this specific demo.
Cue 353 — 00:23:08,846 --> 00:23:14,366
Okay. Again data set repos or releases buckets are for intermediate data.
Cue 354 — 00:23:14,406 --> 00:23:18,606
So again I'm trying to give you an overall idea of the pipeline of how I do data
Cue 355 — 00:23:18,646 --> 00:23:21,286
engineering. Okay. Jobs are runner.
Cue 356 — 00:23:21,326 --> 00:23:24,126
We've already talked about this webhooks or triggers.
Cue 357 — 00:23:24,166 --> 00:23:27,866
So when you get new data it will go and refine.
Cue 358 — 00:23:28,006 --> 00:23:31,966
Basically run this process for you automatically if you don't want to write
Cue 359 — 00:23:32,006 --> 00:23:36,166
manually. So it's a way of running like a job for new data that comes in every day,
Cue 360 — 00:23:36,206 --> 00:23:41,706
or however often the data comes into your specific application.
Cue 361 — 00:23:41,766 --> 00:23:42,666
All right.
Cue 362 — 00:23:44,126 --> 00:23:46,666
So let's talk about evals for a second.
Cue 363 — 00:23:47,166 --> 00:23:50,526
I can show some of these evals in the code if you like.
Cue 364 — 00:23:50,566 --> 00:23:52,566
But there are a couple of things here.
Cue 365 — 00:23:52,606 --> 00:23:54,406
Let me zoom in. This is a little small.
Cue 366 — 00:23:54,446 --> 00:23:57,526
So. Right. So this is the question I asked earlier.
Cue 367 — 00:23:57,566 --> 00:23:59,686
How do I log into Hugging Face from the command line.
Cue 368 — 00:23:59,726 --> 00:24:03,846
So I wanted to go and look at the documentation and give me an answer.
Cue 369 — 00:24:03,886 --> 00:24:10,494
Right. So specifically what I have is stuff like this in here says must include hf
Cue 370 — 00:24:10,534 --> 00:24:13,006
auth login, because that is the correct answer today.
Cue 371 — 00:24:13,046 --> 00:24:19,046
For this it must not include the old CLI command which is the second one,
Cue 372 — 00:24:19,086 --> 00:24:21,346
right. Because that is tagged as stale.
Cue 373 — 00:24:21,406 --> 00:24:24,926
So when you go run this code and you look at how it put this together,
Cue 374 — 00:24:24,966 --> 00:24:28,346
it will actually tag these different types of
Cue 375 — 00:24:29,566 --> 00:24:32,446
test or write what we're calling evals here.
Cue 376 — 00:24:32,486 --> 00:24:36,366
It will tag them so that you know exactly where things break.
Cue 377 — 00:24:36,406 --> 00:24:40,326
Right. So if you get the second response in this case, then you can look at the tag
Cue 378 — 00:24:40,366 --> 00:24:41,726
and say, oh, this is stale data.
Cue 379 — 00:24:41,766 --> 00:24:44,866
That's why it was removed for the second part here.
Cue 380 — 00:24:44,966 --> 00:24:48,326
How do I create a storage bucket hf buckets create.
Cue 381 — 00:24:48,366 --> 00:24:50,926
This is a newer feature. So it's tagged as a newer feature.
Cue 382 — 00:24:50,966 --> 00:24:56,246
So tagging this these questions in this way is very helpful in understanding what
Cue 383 — 00:24:56,286 --> 00:24:59,726
kind of questions you have. Do you have the right mix of different kinds of
Cue 384 — 00:24:59,766 --> 00:25:02,926
questions, right things which are new features, data that might be stale,
Cue 385 — 00:25:02,966 --> 00:25:08,046
etc., things that must be included in the response and things that must not be
Cue 386 — 00:25:08,086 --> 00:25:14,606
included. Okay, so again, this is how I put stuff together to make sure that when
Cue 387 — 00:25:14,646 --> 00:25:21,385
I'm running test against a new data set, or when I'm running test against a new LLM,
Cue 388 — 00:25:21,526 --> 00:25:26,806
if I'm hitting the right types of things as part of this testing process.
Cue 389 — 00:25:26,846 --> 00:25:33,566
Okay, so the overall idea here is to have data to be able to give it to an LLM and
Cue 390 — 00:25:33,606 --> 00:25:36,126
to get correct answers out of it.
Cue 391 — 00:25:36,166 --> 00:25:38,726
Now, we all know that LLMs are non-deterministic.
Cue 392 — 00:25:38,766 --> 00:25:43,246
So this is a way to make sure that our data flows through this entire process and
Cue 393 — 00:25:43,286 --> 00:25:48,246
gives us something that is a much more deterministic answer to what we're trying to
Cue 394 — 00:25:48,286 --> 00:25:50,606
do, right? Or deterministic, whatever.
Cue 395 — 00:25:50,646 --> 00:25:51,846
It doesn't have to be an answer, right.
Cue 396 — 00:25:51,886 --> 00:25:57,346
Deterministic data in, sorry data in and deterministic data or answers out.
Cue 397 — 00:25:58,286 --> 00:26:02,286
All right. And there's a gate here that is also part of this.
Cue 398 — 00:26:02,326 --> 00:26:03,145
And
Cue 399 — 00:26:04,366 --> 00:26:05,846
this is not super interesting here.
Cue 400 — 00:26:05,886 --> 00:26:10,366
But once we pass a certain certain threshold we say this is good.
Cue 401 — 00:26:10,406 --> 00:26:15,326
It's past our 95% threshold for getting the right answer out of this data set.
Cue 402 — 00:26:15,366 --> 00:26:17,206
So then we tag it and then we store it.
Cue 403 — 00:26:17,246 --> 00:26:21,006
Right. So this is this is something that not a lot of people in data engineering do
Cue 404 — 00:26:21,046 --> 00:26:26,145
rigorously enough in my opinion, which is when you work on a data set,
Cue 405 — 00:26:26,286 --> 00:26:32,166
especially in large data set, we need to make sure that we keep versioned histories
Cue 406 — 00:26:32,206 --> 00:26:35,966
of these old data sets around so that we can compare what we did in the past to what
Cue 407 — 00:26:36,006 --> 00:26:39,646
kind of data we have now kind of results we get from that data,
Cue 408 — 00:26:39,686 --> 00:26:44,246
because it will be useful to be able to do that analysis at some point in time.
Cue 409 — 00:26:44,286 --> 00:26:48,486
Again, if we think about what we're trying to do, we put push raw data in,
Cue 410 — 00:26:48,526 --> 00:26:53,126
we refine it, we make it better to get better answers and be able to track this just
Cue 411 — 00:26:53,166 --> 00:26:54,806
like we do in software engineering.
Cue 412 — 00:26:54,846 --> 00:26:57,225
We've tried to version the data as well.
Cue 413 — 00:26:58,126 --> 00:27:05,006
All right. So almost at the end here, but just want to give you the overall
Cue 414 — 00:27:05,046 --> 00:27:07,006
picture of what we're doing here.
Cue 415 — 00:27:07,046 --> 00:27:13,526
So we have an ingest job which takes our docs converts them into parquet.
Cue 416 — 00:27:13,566 --> 00:27:16,625
We put that into some kind of intermediate state.
Cue 417 — 00:27:16,686 --> 00:27:20,446
In this case I'm using a Hugging Face storage buckets which use Xet,
Cue 418 — 00:27:20,486 --> 00:27:24,886
which allows me to very quickly upload and download data when I've changed the data
Cue 419 — 00:27:24,926 --> 00:27:28,766
on the inside of those files, rather than having to upload gigabytes and gigabytes
Cue 420 — 00:27:28,806 --> 00:27:34,846
of data. Then we run the refinery job that takes our data and makes it clean and
Cue 421 — 00:27:34,886 --> 00:27:41,586
better and ready for use. We commit that into a basically a versioned repository.
Cue 422 — 00:27:41,646 --> 00:27:44,166
Buckets are intermediate data storage.
Cue 423 — 00:27:44,206 --> 00:27:49,606
Data sets are versioned permanent data storage or storage for your data.
Cue 424 — 00:27:49,646 --> 00:27:56,686
Okay. And then once we have that we fire a webhook that goes and sees if our new
Cue 425 — 00:27:56,726 --> 00:28:03,486
or updated data set passes our test, our eval gate, are we hitting 95% accuracy
Cue 426 — 00:28:03,526 --> 00:28:08,446
whatever number you decide to choose, okay, then we tag that if it's good and then
Cue 427 — 00:28:08,486 --> 00:28:13,686
we go and deploy that. When I say space, that is this thing that I was running here.
Cue 428 — 00:28:13,726 --> 00:28:18,566
Basically, there's a way that you can run Python applications or you can run a
Cue 429 — 00:28:18,606 --> 00:28:23,366
Docker container for your app, or you can run the jobs or whatever directly onto
Cue 430 — 00:28:23,406 --> 00:28:28,526
Hugging Face cloud. Okay. So but again, you don't have to use our stuff or
Cue 431 — 00:28:28,566 --> 00:28:31,246
infrastructure. You can deploy all this yourself.
Cue 432 — 00:28:31,286 --> 00:28:35,646
You just tell your agent to rebuild this on AWS or Google Cloud or whatever you use.
Cue 433 — 00:28:35,686 --> 00:28:41,286
Okay. All right. So that was our whole loop in terms of what we're trying to do
Cue 434 — 00:28:41,326 --> 00:28:44,486
here. Make that work. We test the refinery.
Cue 435 — 00:28:44,526 --> 00:28:48,226
We have a CD pipeline. We we data.
Cue 436 — 00:28:48,446 --> 00:28:50,846
We have a pipeline for our code.
Cue 437 — 00:28:50,886 --> 00:28:54,566
We want something similar for our data, and we want to make sure that both are green
Cue 438 — 00:28:54,606 --> 00:28:58,006
before we release any code to production.
Cue 439 — 00:28:58,046 --> 00:29:00,846
Okay. So we've already run the demo.
Cue 440 — 00:29:00,886 --> 00:29:07,726
And so I actually when we go do when we go went and ran this specific
Cue 441 — 00:29:07,766 --> 00:29:13,126
demo. This is actually just using, I think GPT-OSS, whatever the latest version is
Cue 442 — 00:29:13,166 --> 00:29:18,566
from a couple of years ago. So what's interesting about that is I wasn't using a
Cue 443 — 00:29:18,606 --> 00:29:22,486
large frontier model to actually do this application, right.
Cue 444 — 00:29:22,526 --> 00:29:24,526
I was just using a very small model.
Cue 445 — 00:29:24,566 --> 00:29:28,786
I think GPT-OSS is like a couple of gigabytes total in size.
Cue 446 — 00:29:28,926 --> 00:29:32,406
But the point here is that if you're using a very large model,
Cue 447 — 00:29:32,446 --> 00:29:37,446
whether it's one of the frontier models like Sonnet or Opus or ChatGPT or whatever,
Cue 448 — 00:29:37,486 --> 00:29:40,766
oftentimes you don't need that kind of sledgehammer for something like this.
Cue 449 — 00:29:40,806 --> 00:29:44,886
You can use a much smaller model, and why would you want to use it?
Cue 450 — 00:29:44,926 --> 00:29:46,766
Number one, it'll be much faster.
Cue 451 — 00:29:46,806 --> 00:29:48,966
Number two, it'll be much cheaper, right?
Cue 452 — 00:29:49,006 --> 00:29:53,446
So if you can achieve the same goal of whatever you're trying to build in your
Cue 453 — 00:29:53,486 --> 00:29:57,366
application, like in this one, it's going to ask the documentation or go ask what we
Cue 454 — 00:29:57,406 --> 00:30:02,126
have and get answers, correct answers very specifically.
Cue 455 — 00:30:02,166 --> 00:30:05,846
But do we want to burn a lot of expensive tokens to do that?
Cue 456 — 00:30:05,886 --> 00:30:12,126
Probably not. Right. So again, when when we refine, when we refine the data and we
Cue 457 — 00:30:12,166 --> 00:30:16,086
clean it, and we use these good data engineering practices to build this kind of
Cue 458 — 00:30:16,126 --> 00:30:18,766
stuff out, you're able to say, you know what?
Cue 459 — 00:30:18,806 --> 00:30:24,526
Actually, we don't need a very sophisticated high-end LLM to be able to do this
Cue 460 — 00:30:24,566 --> 00:30:27,966
because we're well, how's the saying go garbage in, garbage out,
Cue 461 — 00:30:28,006 --> 00:30:29,606
right. What's the opposite of garbage in?
Cue 462 — 00:30:29,646 --> 00:30:32,046
Garbage out, I don't know. Anyways, you get the idea.
Cue 463 — 00:30:32,086 --> 00:30:38,886
So when we clean this up, we can use a much faster and cheaper model to be able to
Cue 464 — 00:30:38,926 --> 00:30:44,206
do the same things. Okay, and I don't know about you, but I don't think tokens have
Cue 465 — 00:30:44,246 --> 00:30:46,406
really gotten that much cheaper in the last year.
Cue 466 — 00:30:46,446 --> 00:30:48,766
I don't feel like they they will for a while.
Cue 467 — 00:30:48,806 --> 00:30:51,446
Okay, not until the price of memory comes down dramatically,
Cue 468 — 00:30:51,486 --> 00:30:53,126
but that's a different story. Okay.
Cue 469 — 00:30:53,166 --> 00:30:54,066
All right.
Cue 470 — 00:30:55,566 --> 00:31:01,966
So again, one of the reasons why we version our data sets as we add new things or
Cue 471 — 00:31:02,006 --> 00:31:08,206
refine it is to make sure that we're able to go look historically to see what what
Cue 472 — 00:31:08,246 --> 00:31:12,846
changed in our data. Why are we getting worse results with this updated data set or
Cue 473 — 00:31:12,886 --> 00:31:14,846
with this new data we've added in it?
Cue 474 — 00:31:14,886 --> 00:31:17,646
If you have the old data set, it's version, you can go and do a comparison.
Cue 475 — 00:31:17,686 --> 00:31:21,726
You can figure that out, or you can tell your AI agent to go and figure that out for
Cue 476 — 00:31:21,766 --> 00:31:28,946
you. Okay. All right. So again, what we tried to do here is
Cue 477 — 00:31:30,446 --> 00:31:37,286
in the short demo we did when we looked at this, right, the crude data or
Cue 478 — 00:31:37,326 --> 00:31:42,445
just the raw data unprocessed, it gave us the wrong answer after we had processed
Cue 479 — 00:31:42,485 --> 00:31:44,586
it. It did give us the correct answer.
Cue 480 — 00:31:45,366 --> 00:31:49,965
One of the things that significantly helped to get the right answer here was to make
Cue 481 — 00:31:50,005 --> 00:31:54,886
sure that we didn't include older versions of the documentation as part of this.
Cue 482 — 00:31:54,926 --> 00:31:59,525
So a common question at this point is, well, why didn't just you just use the latest
Cue 483 — 00:31:59,565 --> 00:32:02,566
version anyways? Right? That would be an easy fix.
Cue 484 — 00:32:02,606 --> 00:32:04,525
And that actually does work, right?
Cue 485 — 00:32:04,565 --> 00:32:10,406
So you could basically do no refinement and just say, let's just use the latest
Cue 486 — 00:32:10,446 --> 00:32:14,286
version of our documentation so that we know we always get the right answer,
Cue 487 — 00:32:14,326 --> 00:32:19,166
at least for these class of questions that we'll get that we need to get answers
Cue 488 — 00:32:19,206 --> 00:32:24,445
for. Okay. But if you think about that, when you make that decision as an engineer,
Cue 489 — 00:32:24,485 --> 00:32:29,566
so you just use the latest data set, what you're doing is you are filtering that
Cue 490 — 00:32:29,606 --> 00:32:31,766
data, right? You're not letting the machine do it.
Cue 491 — 00:32:31,806 --> 00:32:35,646
You're doing it conceptually and also physically yourself.
Cue 492 — 00:32:35,686 --> 00:32:38,766
Right. And then also brings up a whole host of other questions.
Cue 493 — 00:32:38,806 --> 00:32:42,646
Are the documentation the latest documentation 100% complete?
Cue 494 — 00:32:42,686 --> 00:32:47,806
Is there useful stuff in old documentation where you have like migration notes or
Cue 495 — 00:32:47,846 --> 00:32:50,846
what's changed in this new release of the software?
Cue 496 — 00:32:50,886 --> 00:32:53,705
That stuff is actually extremely useful.
Cue 497 — 00:32:53,926 --> 00:32:58,926
Again, for this specific example, it's extremely useful because it will have extra
Cue 498 — 00:32:58,966 --> 00:33:03,855
notes that may not be in the actual documentation that you may want to provide into
Cue 499 — 00:33:03,895 --> 00:33:09,005
the LLM as a chunk, so that it can give you a correct answer or have better context
Cue 500 — 00:33:09,045 --> 00:33:11,546
to formulate a good answer. Okay.
Cue 501 — 00:33:12,686 --> 00:33:14,806
All right. So we're over time now.
Cue 502 — 00:33:14,846 --> 00:33:17,646
So I'm going to we've already talked about all this stuff.
Cue 503 — 00:33:17,686 --> 00:33:18,946
So we're at the end.
Cue 504 — 00:33:20,246 --> 00:33:23,726
Thank you for staying a few extra minutes for this presentation.
Cue 505 — 00:33:23,766 --> 00:33:25,406
And I know it's right before lunch.
Cue 506 — 00:33:25,446 --> 00:33:27,586
So we'll take questions here over there.
Cue 507 — 00:33:28,045 --> 00:33:31,406
But please don't ding me and give me a bad review for cutting into lunch.
Cue 508 — 00:33:31,446 --> 00:33:35,166
It's surprising that if you run into lunch, you get worse reviews,
Cue 509 — 00:33:35,206 --> 00:33:37,525
right? I don't know why that is, but but please don't do that.
Cue 510 — 00:33:37,565 --> 00:33:39,846
I hope you enjoyed the talk. Please connect.
Cue 511 — 00:33:39,886 --> 00:33:43,806
And again, if you're interested in having this as a workshop or a fine tuning
Cue 512 — 00:33:43,846 --> 00:33:45,806
workshop at your company, please reach out.
Cue 513 — 00:33:45,846 --> 00:33:47,005
And this is part of what I do.
Cue 514 — 00:33:47,045 --> 00:33:48,785
Okay. Thank you all for coming.
.tessl-plugin
skills
talk-arcolano-autonomy-and-trust
talk-aroussi-decision-context
talk-bevin-agent-assisted-instrument-design
talk-boeckeler-context-lifecycles
talk-braun-context-graphs
talk-carmont-evolving-agent-harnesses
talk-chanezon-gordon-agent-experience-evaluation
talk-cox-secure-agentic-workflows
talk-cunniffe-reusable-agent-platform
talk-davis-agent-tool-policies
talk-debois-concurrent-delivery
talk-dzialets-team-automation
talk-fahmy-harness-api-design
talk-finster-continuous-delivery
talk-fowler-regenerative-software
talk-gavrilescu-factory-architecture
talk-graciolli-amin-verification-aware-programming
talk-gruenbaum-reverse-graph-engineering
talk-hassan-agentic-engineering
talk-hossain-enterprise-agent-adoption
talk-huntley-patch-ready-factories
talk-jain-verifying-intent
talk-katsioloudes-montei-agentic-lifecycle
talk-knox-harness-engineering
talk-kovac-deterministic-memory
talk-lee-perkins-code-readiness
talk-lorenc-machine-speed-security
talk-lubken-software-of-one
talk-meijer-software-specialization
talk-meppiel-versioned-agent-contracts
talk-moss-concurrent-agent-risk
talk-myers-understanding-agent-generated-code
talk-ocytko-scaling-agentic-engineering
talk-oza-automating-design-quality
talk-panzica-agentic-sdlc
talk-pari-patel-agent-awareness
talk-podjarny-trustworthy-software-factories
talk-poulter-factory-improvement
talk-pratik-patel-refined-rag-data
talk-re-ferre-development-platforms
talk-sadogursky-factory-of-factories
talk-shah-training-agent-skills
talk-sharma-mohapatra-evolving-intent
talk-singhal-operating-agents-at-scale
talk-stack-incident-dossiers
talk-tyler-team-coding-benchmarks
talk-wotherspoon-agent-execution-environments
talk-yegge-ai-powered-factories
talk-zuber-leadership-through-coding