Transcript — Using skills to pay the bills: graduating from solo hacks to a team workflow

⚠️ Speaker-label warning. This transcript was produced by automatic speech-to-text and has no per-speaker labels. The vast majority is James Moss delivering the talk. There is also:

• An unnamed MC / colleague who introduces Moss at the start and closes the session at the end.
• Four unnamed audience questioners during Q&A.
• Boris Cherny (creator of Claude Code) is quoted by Moss from a separate forum — not present.

When attributing words, default to "Moss said..." for the main body. For the intro/outro use "the MC said...". For Q&A use "an audience member asked..." and "Moss replied...". Do not invent attributions.

The transcript also contains many speech-to-text artifacts and likely-garbled terms. Notable probable corrections (preserved verbatim below):

• "Wilming school" → likely "Grill me skill"
• "Sneak" / "snake" / "Snyk" → "Snyk" (security scanner)
• "mini sha loot tack" → likely "Shai-Hulud attack" (the late-2025 npm supply-chain incident)
• "AKM" → an unclear package manager name (possibly a Microsoft-affiliated tool)
• "fabric" (in Q&A) → likely "package"
• "gooey" → "GUI"
• "11th list" → likely "11th letter" (referring to K in the alphabet)
• "ed skills" → likely "old skills" or similar
• "Wilson quote" → almost certainly the Adam Savage quote

Section: Intro & framing

Thank you. For. Coming. This is. A. Special talk. From my colleag. Ue James. Moss. We work together at Tessl. James is a member of. Technical staff. Which means he's. A software engineer with a priority complex. Just kidding. I have been liking this. Eventually we'll all just be employees. And I look forward to that. Time personally. So James is going to be talking about a problem that I would be surprised if you haven't encountered yourself. Everyone in a team kind of uses. Agents. A little bit differently. James is going to talk about some of the tension points when you're disparate agent workflows come together to try and work on the same piece of software. So with that, over to James. Moss.

Hello everyone. Hope you guys are enjoying our native Devcon. So far, hope you had a little bit of a break. Some coffee, brains warming up. We're starting to feel a bit more engaged. So a little bit about me. My name is James Moss. I spend probably more than two decades working in the industry across the stack doing lots of different things. Right now I'm a software engineer of technical staff at Tessl, where I work on registry product amongst many other things. As you do. At startup. S.

So using skills to pay the bills. If you didn't read the abstract and you just read the title of the talk and you came and you thought today I was going to be sharing a get rich quick scheme, here it is. Maybe it works, maybe it doesn't. You're probably going to be disappointed in the rest of the talk and I've got to give you today. Maybe your takeaway is some priceless nuggets that you can take home and use yourself instead of this crazy skill. So yeah, we're going to talk about adoption and management of skills within the enterprise. So there's quite a different challenge to using skills within a person five coded project that requires a different set of practices.

Section: The three-part plan

So plan for today, we're going to cover three things skills sprawl, which I'll talk about shortly, which you might have heard of before skills or software. So building on similar things that Guy was talking about this morning in his keynote and then a small piece at the end around the context development life cycle.

Section: Skills sprawl — definition

So let's kick off talking about Skills. First of all, maybe a term you haven't heard before. If you think about skills and authoring them, the barrier to entry is super low, right? They're mostly just text files. Sometimes you might put a couple of scripts in there with them. It's super easy to add your own new ones or install third party ones from GitHub. They're also pretty powerful as well, right? Like it's very easy to write a small amount of text and influence how the agent works in quite a powerful way.

And so because of this, teams are facing this kind of Cambrian explosion of skills within their organizations. I mentioned in his slides this morning we have 2 million plus skills across GitHub across 44,000 repos. It's an insane amount. That's obviously just the public ones as well. I'm sure it's even higher in terms of internal private repository skills as well.

So yeah, we're seeing skills spread across lots of different locations. So GitHub, as I mentioned, cross things, marketplaces and also on developers local machines as well. It's hard to get visibility around where skills are being used. And if you can't track that, then you can't measure that and you can't improve things. So this is a real problem right now that we're hearing from customers.

I collected some quotes that I want to kind of just quickly go through with everyone here and just to kind of outline this. Problem. First up. We have a plethora of skills distributed across so many repositories that it's become incredibly difficult and almost impossible to get your head wrapped around how to unify, centralize and properly distribute skills across all teams. From a lead architect and AS services company. A couple more. It's a free form right now for the most part. Everyone's been doing their own thing. It's the wild west, right? Another couple of engineers there, right? These are experienced folk from the industry, from companies you've definitely heard of. And this is an industry wide problem right now.

So a quick show of hands who here kind of agrees with these statements. Who here doesn't have a clear picture of every scale they use across your team. Amazing. Lots of hands. I was worried no one was going to say this in my talk. So yeah, hopefully lots of you are recognizing this problem already. And you're facing the pain of that already.

Section: The five failure modes

So how does this rule manifest actually and impact humans and the agents that we're constructing? So there's a couple of different failure modes that we're seeing.

The first one is overlap. So this is where you might have multiple teams all building the same thing in isolation without realizing that each team might build their own version of the skill. It might achieve the same thing. You might have generally the same outcomes, but it's done in a different way. You're having multiple kind of wasted effort there. So that's overlap.

Next up, we often see drift. So this is where newer skills are shipping. And teams aren't keeping up with those newer versions. Right. And a good example here recently is Matt Pocock. Hopefully you've all heard of him. He's great educator and published a bunch of great skills. He has a Grill me skill. That's where you kind of do an interview with the agent and you kind of get a shared understanding. He recently published a newer version of that Wilming school called Grill with Docs, which kind of expands on there and has a lot more detail to it. I'm sure there's lots of folks that don't realize that's been released and they're still using that older version and haven't updated. So that's drift.

We also have problems around activation or lack of activation, right? Lots of people are producing these skills. Are they being used by the agents? Are they being used by humans? There's no real way to understand that and know that right now. You probably don't have that disability. Yeah.

Rott, so I mentioned this in his talk as well this morning. You might have a skill that describes code or a code base or a process or even a workflow. And those two things can cook your graph since. So your skill never gets updated, things change. In those instances, having an outdated skill can often be just as bad as having no skill at all.

Next up is overloading. The last one here. And this is one you might not be aware of. This is having too many skills in a single repository. So right now you want to know that there is actually a limit to the amount of skills you can have with most coding agents, often expressed as a percentage of the context window. So if we think about our skills, the name, the description gets injected into the start of every context window. A sluggin nose about their skills and knows when to use them as a decision. Activating them. What can happen here is undefined. Most agents will truncate that description, which means you end up in a situation where your skills might not be activated because the description has been shortened and keep its information that the modern companies know about that skill and not there anymore.

So these are the major failure modes that we're seeing.

Section: Tessl's sprawl-detection product

So we set out to detect and actually solve these problems for our customers. And so Tessl's mentoring. We connect to your GitHub org, we pull in all the skills, we find across all your repos. We'll let you kind of drill into how they're being used. So you can break it down by repo, you can break it down by skill. You can see those first party skills, those third party skills. You can see token usage on their skills. And we're also able to surface those findings as well that I talked about. So the previous slide, those different failure modes, we're able to find those, surface them to produce a report and then you can go and fix that and decide what we want to do.

We're also able to lean on our public registry and if we're able to find you a third party skill that you're using within there will also surface any reviews or email scores or security scans we've also done on that scale.

This is currently in private beta, very, very new. If you're interested, come and find me. I'm working on the Tessl booth for the rest of the day. I'm more than happy to chat. We've got a quick demo and run you through this if you're having the same problems as most people seem to be.

So a quick recap on sprawl uses exploding, right? We need to start measuring how we're actually using skills across our organization. And we need to track that over time so that we can improve that and resolve those issues. So that's a broad recap.

Section: The agentic equation

Next up, I want to talk about skills of software. So this is kind of building on a lot of the things the guy talked about this morning. I want to give you some actionable things that you can take away and use it within your own teams and you're in organizations today. So I'm not going to go into lots detail about what makes an individual skill work. Hopefully many of you in here are already familiar with that. There's lots of resources on there. But I want to talk about the kind of ecosystem that exists around.

Before we dive into that, I want to talk about the agentic equation. Right? So if we think about an agent is a function of the modern, the harness and the context, right? Changing any one of these three pieces has an impact on how the agent works and its output and the quality of its out. Put. Most of the time we're probably not going to change the model of the harness so much that's harder to do. Or maybe we're kind of tied into using a certain model or a certain harness within the organization we're working in. And so context is effectively the best lever that we can pull to affect the outcome of the agent here. And it has a pretty big impact as well, right? I'm sure most of you are familiar with this, right? You can do a lot in terms of what you're doing instructions and the skills you use and the prompting you use and the kind of hooks you have set up.

So within context skills again feel like an important powerful piece because they can encode business or domain specific logic that the model won't have been trained on. So you can give them all of that knowledge that it won't otherwise know about.

Section: Skills-as-software — decompose

So given all of this, thinking about this, what can we do to make skills as effective as possible? So first one, nice and easy. Hopefully we're already doing this. Don't write monolithic skills, right? Don't write huge epic skills. Think about decomposing breaking skills down into smaller pieces. Most agentic coding tools for the use of plugins. If you're not familiar with plugins. Plugin is just a group of contexts related together that can be installed into your ejected coding tools. Agents can then pick and choose which parts of the plugins they want to use because you're breaking down your skill into smaller ones. You can use descriptions for each of those individual pieces and so the agent is more likely to accident those individually rather than trying to stuff a load of kind of keywords into a bigger description of a larger scale.

You can also end up building kind of pretty complex interconnected skills as well. So Guy mentioned this morning that skills from cool skills and so you can have one skill that has maybe been looked by a human that sets up the agent to call a bunch of other skills. An example of that recently is we built a UI skill. Tessl, which enabled us to build user interfaces and browser. So we would take a figma design. We feed that into one skill and I was able to break that finger design down into various components and then hand that off to sub agents using other skills to implement those. So we have a specific skill for building kind of low level components like buttons, a separate one for building high level components, things like pages and another one for the kind of those things and then we don't think things like forms perhaps compose buttons, text inputs, that kind of thing. And so the agent was able to do all of these, but the powerful part about breaking them apart is that me as a human, I could go in and if I wanted to just build a small widget, I could just invoke the kind of low level sort of like button level scale that describes how to build sort of generic components if we wanted to. So that's decomposed.

Section: Skills-as-software — extend, don't edit

Extend. So skills are just text, right? And they get vended in into your repo as well. So you have a copy of bunch of all these skills. Very tempting to just edit them directly. You've got some issue with the way the agent is acting after using your skill. I'll just edit that right and get repo rate. Please don't do that. Your teammates won't like you.

What you want to do here is extend third party skills. So let's say you've got a great code review skill that you like to look on, but it's not tailored to the technology stack using or best practices you have employed in your company. Write your own first party skill and work with that skill. Agents have a pretty good understanding of kind of intent here and what needs to sort of override each piece and they can fill in the blank and extend.

And then lastly thinking back to our kind of CS 101. If folks remember that, thinking about the solid principles, s is single responsibility principle. It's important for skills too, right? So don't overload that skill. Again closely coupled to decomposition.

Section: Skills-as-software — avoid global skills

Next up avoid global skills. So what do we mean by this? Right? This is the works on my machine problem. Skills that are installed on a developer's machine live outside of the repo. So if a skill lives on someone's local config rather than the repo, the agent, sorry, the agent's behavior depends on who's running it. Two different people run the same task against the same code base and get very different output because one has the skill and the other one doesn't or it has a different threshold. Horrible to debug. Try and work out what's happened, right? The skill isn't in the diff. You saw the skills in the diff, it's not in the review, it's not ancient. And then divergence kind of has no paper tray up here. It's really hard to track down. The fix here is the same instinct is committing kind of your log file and traditional code packages. Skills that affect output should be version controlled alongside the thing that they're acting on. So not flowing around and use this home directory.

A colleague recently attended the small forum with Boris Cheney creates employed code. And he had this to say. So we ban any local setup. All agent workflow improvements, hooks, skills, etc. Must be checked into the repo for everyone. So the Boris is doing it. He's following my guidelines already. So amazing work, Morris.

Section: Skills-as-software — automated skill reviews

Next up is automated skill reviews. So I don't think you or probably your agent wouldn't commit code without running it through lintza. We often ask agents to write tests for us for kind of a single modular code. Skill review is kind of similar to these two processes. They're quick to run. You can run them locally after making changes. The content series skills kind of get a quick feedback loop. You can also run them in CI as well to kind of gate quality regressions and sort of failed build if schools don't meet that certain level.

Here in the screenshot, hopefully it's not too small. You can see I've run the skull review on skill I shared at the start. So lots of green tips. It is a valid skill. The line count is not too long. The front master is valid. All good. And our skill review also has another piece to it which uses an llm as a judge, which is not so happy. It's giving me a score of 20% here saying that I definitely shouldn't. It's got inherently dangerous financial operations. So, you know, we have a certain level of gating for skills within Tessl is definitely isn't getting published. It's not getting past it.

Section: Skills-as-software — publish to a registry

Next up published to a registry. So right now I imagine most of you are here are probably loading your skills directly from repos on github sort of via cloud marketplace or perhaps using something like npx skills command. And that's fine. That's the path of least resistance right now. But the registry adds a lot more on top. Right. And you have this one sort of truth. Everyone's pulling the same notion of the scale, same conversion.

Right now there's a, it's kind of an explosion of skills on github that are forks. Right. So someone fork to skill. It's from a different place. They've changed something slightly. It's very easy to end up with two skills that name the same thing and look like they do the same thing, but they're not in fact the same skill.

So registry also allows you to stay current safely. So know when there's updates, review diffs roll back that kind of thing. Discoverability is a big piece if you think about skills for all overlap is a big concern. You know, having a centralized registry where all this stuff is can help people find those skills and not right their own as well as kind of maybe promoting skills to you that you want to use.

And lastly, there's an interesting one that we're hearing from some customers as they're rolling out skills to non tech folks. This girl living inside private github repos and they have to provision github seats for those non-technical users. So even though they're not using any of the features of github, they're having to pay how many bills in a month right now for users to be able to access those skills prior to putting a registry in front of where your source code lives helps negate that kind of issue.

Registry is also a great enforcement point, right? It gives your surface to be able to put in a place governance and enforce company policies. Kind of examples of those things like improved skills only. Perhaps you allow all your first party skills, but only a small subset of third party skills. Is the kind of risk profile within your company. Similar piece around security and kind of bad actors. Sneak, we do security scan, sorry, a Tessl, we use snake. We plug the whistle to do security scans of all the things that our budget to the registry, right? And so you might want to enforce that only kind of highest level security checks pass before skills can be installed.

And then finally minimum release agent, right? Like I think this is definitely quite personal right now for those that are not aware you can set a release age of say a few days and any badges that are newer than that can be installed. This is especially permanent for us. Tessl last month three or four weeks ago we had the mini sha loot tack which affected a bunch of packages that we depend on with big Tessl stack fans at Tessl. And the malware here was notable because it was basically just flight gene tag. It was notable because it had a level of persistence where it added a global clawed code hook to your machine to try and keep itself installed on the user's machine. Right. So another good reason not to use any kind of global state coding skills. We were affected by this because in our node package manager we haven't released age of a few days. The Tessl team pretty quickly cleaned it up. So there's no way our debts could have installed that.

This is not such a big thing right now in terms of some of the skills on registry. I think open core maybe is definitely an issue right now. Guy mentioned this morning it was a 20% of all the skills in there are malicious. We're going to see that expanding into other registries, other skills as well. And it's not one surely change. But if you're loading it straight from github, what's in the way, what stands between that code being on github, it compromised and being installed into your machine versus having something like a registry in the middle that is able to kind of save you there.

Section: Skills-as-software — don't lock to one agent

Next up don't lock your skills to one agent. I grabbed a screenshot from sales skills repo. Tickle the project paths. How many are there? So many, this is also only up to k as well which is what the 11th list is in the alphabet. There's many many more right. There's some good people that are using the dot agents kind of pattern for putting their skills inside. But kind of this is crazy right? And so if we think about it, the agents we're using in six months might look nothing like today's right. The cost of tokens is going up open weight models are getting more and more kind of traction and you don't want to be just hiding to one coding agent. Right. Skills are effectively your durable asset. The agent is just the room type. You want to keep them postable and you don't have to rewrite your investment every time the land landscape shifts. What you want to do experiments with new tooling. Also why registry matters as well especially at package manager can abstract away a lot of this pain and having to put these things to different places you've read it once and then the manager will take care of installing it to whatever agent you're using.

Section: Skills-as-software — make context a team asset

Next up making context a team asset. Not just a personal setup keeping things dry so one source of truth having shared ownership not being one person, one team own everything super important making them safe to contribute right if people feel like a skill is highly dependent on one by many others in the company they might not feel safe contributors. So putting in place those kind of CI checks I mentioned gating and creating safe space where they can edit skills, create skills without feeling that they're going to break the kind of internal ecosystem for many folks super important.

Section: Skills-as-software — measure with evals

I'm going to touch on emails finally measuring with evals. So I like to think of evals as a bit of experiment and treating skill success as a measurement sorry as an experiment and if you can't measure it again you can't improve it. I found a couple of funny posts from reddit and linkedin. Here we've got a claim that they built on claud skill that can reduce stops overthinking and cuts token use it by 60 to 80% seem pretty crazy. I wouldn't claw to think more nonetheless. 44% increase in coding accuracy right four rules from kathy added to your claude md and coding actress he jumped from 65 to 94% I didn't really know what coding actually means but it's great to see my uptick. This one 20,000 tokens added to every prompt right this guy here is saying he's got this magic prompt that kills 90% of production bugs before they're red. Is it bug if it never existed? I don't know that sort of like hydrogen all these kind of claims make me think about this Wilson quote from Adam Savage and myth busters fame hopefully you know who he is. I think this quote now the difference between screwing around in science is breaking down it's kind of more important than ever in a sort of vibe space world that we're living in.

So to me emails is a bit like writing it down. If we go back to the agentic equation eval is likely change any part of it right the model, the harness, the context and measure actually happens to help. You can run emails against a single skill or multiple skills and you can run them against a real code base to see what holds up and what breaks. They also answer questions that might be difficult to answer by doing kind of manual testing. So things like will this change rating still worse swapping different models out in many different models? Does the skills still work? Do you even need the skill? Is the agent sorry has the model just got better and you no longer need to worry about kind of documenting some of these processes that you are with ed skills.

So quick recap on skills and software decompose do your reviews. Make sure you implement a registry and have policies around that that kind of fit your company and the things that you care about. Team ownership make it safe for everyone to contribute and then finally using emails and measure that success with experiments.

Section: The Context Development Life Cycle (CDLC)

So lastly I want to touch on the context of animal life cycle. Everything you do to keep code healthy already has a direct equivalent for context right you probably just haven't really been doing it. Yet. If you think about skill reviews, they're like linting and unit tests. Maybe that's a bit more like end-to-end tests. Code registries are kind of like sorry context rotations like code registries. And when it comes to authoring skills the same reflexes that you apply with code also play to context and skills right you compose things together you avoid global state. All of those things matter. And lastly same things with human factors for code right? We don't have silo knowledge avoid that kind of bus factor. So all those same things apply.

Section: 2005 PHP story

I want to leave you guys on a bit of a story. I want you to think back to 2005 if you are able to remember that if you're sadly old enough to remember it. Myspace was in its heyday. Mr. Brightside by the killers was top in the charts pre-i phone and no iPhone is pre-GitHub pre-containers pre-dev ops. I was a fresh face developer at university writing php and my very first gig.

At that time php had no package manager right it was seven years before composer came around. And so if we wanted to install things we'd go into sourceforge, we downloaded zip, we'd extract that event on that into our code base. We would never do any kind of security review. We're not looking at that code and I was thinking hey does this have a vulnerability in it? We're just like cool this is a date library. We want to have an arch, right? There was no semantic versioning that hadn't been invented yet. It was whatever the authors kind of win was to updating that deployment kind of made me wince a little bit thinking about deploying. So we used to use FTP and we would we would copy files across on ftp in straight into production which it's mad to me now. We went to monsters. We used SFDP. So this is crazy right? Like we would never dare to do any of this kind of stuff now and it took the industry about 10 years to sort of fix all of this right we invented at least for pHP we went to the package manager we had semba we had log files registry CI test dependency scanning sign release of all this stuff right all this stuff that we now kind of take for granted. Youd never imagine driving a file into project so we were never doing it as our data practices anymore but it does feel like we are doing them a bit with skills and context right now like in some of the ways we're working where we're skipping package managers we're installing directly from source and we're not we don't have these kind of CI things in place right so the good news is we already know how the story ends right we don't have to spend another 10 years coming up these processes of rediscovering those we can apply what we've learned with the sdlc to the CDLC right we just have to make sure we're not making the same mistakes again. Thanks.

Section: Q&A

MC: You good yes we have four minutes more questions for James?

Audience member 1: Thank you so I've been using the Tessl registry and I was curious about now you're migrating everything flag is understand from skills and so if I have multiple skills do I have to package them into fabric in the plugin should I keep them separate? What's the suggestion here

Moss: classic answer it depends I think we are planning on allowing suspended upload individual skills and they were kind of focusing on plugins right now it comes down to your needs right like my example there I was talking about a bunch of skills that related so that makes sense to package them you might want to also package a bunch of kind of related plugins that are broader how do you have a bunch of plugins that everyone uses right almost like a standard library for for skills across your company and you want everyone to install that and you want to just give them one dependency that they can install and keep things up to date so yeah as I say it depends it depends on your use case we want to support both ways whatever is kind of flexible for yourself.

MC / Q-runner: Because we have a very aggressive warning from declining from the client you have to move the flight.

Audience member 2: I've looked to hear more on the how you share this with non technical people in the company because like main backstage is using jira and the update tickets and using skills to help with that and you want to share that with the other half of the business. How do you do that

Moss: this is a huge problem right now like this is probably again one of the bigger questions we're hearing from customers I don't think is a good answer expecting folks to you know start running CLI commands in the tunnels locally I think is going to be difficult. Yeah definitely good answer for that one it is a real problem people don't again we don't want to be in a position where we are passing into zips and files around getting focused and still that I think what we'll probably end up having is some sort of gooey imagine for most folks and probably around us so I don't know I'm not committing to that.

Audience member 3: Thanks the talk it's excellent what do you think to AKM as a solution for some of the problems that mentioned

Moss: great yeah I've played around with it a bit very very good definitely music yeah if that's what you want to use for those that don't know package manager from I think it was signed by one guy at Microsoft now kind of come on your umbrella umbrella of the whole microsoft yeah pretty good definitely is a few if that's going to work for you.

MC: Anyone else? Yes.

Audience member 4: Thanks how do you recommend expressing dependencies between skills particularly skills that are not in the same package so if you have a kind of generic set or specific set

Moss: because there isn't a standard way to understand it the moment dependencies something that I think everyone's a bit scared to tackle guy mentioned this morning again is keynote around dependency management and resolution what pain that is I didn't want to sneak there's loads of folks that didn't work with sneaker tassel today but face that pain and they understand that I think ultimately it comes down to maybe having something like independencies I think we're also in a position where maybe there's something better than that right like we're moving to more agentic worlds pale fancies and maybe an artifact of this world where things tend to be more predictable and deterministic and so maybe just the agent being able to look at the skill and see hey I need to use this other skill by reading it I mean that was great.

Section: Wrap-up

MC: Thank you guys that's all we have time for but lucky for you James will be easy to find if you want to chat with him he'll be at the Tessl booth today and tomorrow. Yeah on birthdays yeah come by and chat James thank you all so much the next talk will be in exactly 10 minutes 11:45 you'll get a preview of stack overflow for agents if that doesn't pique your curiosity I don't know why you're here thank you very much.