Lesson 4 — Reviewing skills for security

The last three lessons put numbers on one question: is this skill any good? Review evals and task evals measure quality. This lesson is about a different question your registry answers before anyone runs a line of your code: is this skill safe to install?

When you publish a plugin, Tessl doesn't only score it for quality. It also runs a security review: every skill is scanned for known vulnerabilities with the Snyk scanner. The scan runs server-side, automatically, the moment you publish; you never invoke it. It costs 100 credits per publish, and a workspace can turn it off. What it looks at is the code a skill bundles, meaning helper scripts and their dependency manifests. A skill that pins an outdated library with a published advisory gets flagged exactly the way Snyk would flag it in any repo.

The result is a security score, shown in the registry next to the quality score, so anyone deciding whether to install can weigh risk before pulling your skill into their repo. Each finding carries a severity; critical and high are the ones that gate installs.

It informs, it doesn't block. Tessl is built to give you flexibility, not to stop you. A scan surfaces risk; you decide what to do with it. Scanners produce false positives, so every finding in the registry has a Report Incorrect Findings link, and a wrong call can be corrected.

Keep the two halves straight, because you'll play both today. On publish, the scan runs and the score lands on the registry page (the author's side). On install or update, critical or high findings prompt for permission before the skill lands (the consumer's side).

What you'll build

By the end you'll have:

• A commit-conventions plugin published with a real, scannable vulnerability in it.
• A security finding you read on the registry, traced back to the exact line that caused it.
• The install prompt felt from the consumer side, and a clear-eyed decision about what to do with it.

Start with a plugin that has something to find

The security review only earns its keep when there's something to find. Start from a commit-conventions plugin that has grown a small Node helper, a script that parses the staged diff, where that helper depends on an old library with a published advisory:

commit-conventions/
├── .tessl-plugin/
│   └── plugin.json
└── skills/
    └── commit-conventions/
        ├── SKILL.md
        └── scripts/
            ├── package.json
            └── parse-diff.js

The manifest is an ordinary plugin manifest:

{
  "name": "my-workspace/commit-conventions",
  "version": "0.1.0",
  "description": "Proposes a Conventional Commits message from the staged diff.",
  "skills": "skills"
}

The vulnerable surface is the helper's package.json:

{
  "name": "parse-diff",
  "version": "1.0.0",
  "dependencies": {
    "minimist": "1.2.0"
  }
}

[email protected] has publicly known prototype-pollution advisories, fixed in later 1.2.x releases. It's the kind of easy-to-miss pin that ships in real skills, and exactly what the Snyk scanner flags on publish.

Why bundle a known-bad pin on purpose. A clean plugin produces a clean scan and teaches nothing about the security review. Shipping one real advisory lets you watch a finding land on the registry, then meet the install prompt from the other side.

Publish, and read the security score

You've met tessl plugin publish already. Publishing does more than lint the plugin and run a review eval; it also hands every skill to the Snyk scanner:

tessl plugin publish ./commit-conventions

That publish kicks off the security review server-side (the 100 credits). Within a few minutes the plugin's registry page shows a security score beside the quality score and lists each finding with its severity. The [email protected] pin from the fixture lands as a prototype-pollution finding, critical or high, the severities that gate installs.

Pull the plugin's registry details from the CLI with:

tessl plugin info my-workspace/commit-conventions

Then open the registry page to read the finding in full. Each one carries a Report Incorrect Findings link; if the scanner is wrong about your code, that's how a false positive gets corrected.

The scan is on the version, not the name. The score belongs to the exact published commit. Fix the pin, publish again with --bump patch, and the new version is rescanned from scratch.

Install it — and decide

Now switch sides. Move to a fresh repo and install the skill you just published, the way a teammate would:

tessl install my-workspace/commit-conventions

Because the published version carries a critical or high finding, the install doesn't just proceed. Tessl surfaces the security warning and asks for permission first:

⚠ my-workspace/[email protected]: 1 high security finding
  (prototype pollution via minimist). Proceed with installation? [y/N]

Installation is never blocked; you can always continue. You have three honest choices:

• Proceed, if you've read the finding and accept the risk.

• Pin the scanned commit, so you install the exact version you reviewed on the registry:

  tessl install my-workspace/commit-conventions@<commit-sha>
  

• Pre-accept the warning where there's no human to answer the prompt, such as CI:

  tessl install my-workspace/commit-conventions --accept-warnings
  

--accept-warnings behaves the same way on tessl update when a new version introduces findings.

A prompt, not a gate. The review's job is to make sure the risk is seen, not to make the decision for you. In automation --accept-warnings keeps the build moving, but someone still has to have read the registry page first.

Verify the review, end to end

Four checks prove you've seen the review from both sides:

• The plugin's registry page shows a security score next to the quality score, with at least one finding listed and a severity attached.
• The finding traces to a real cause, the [email protected] pin in the bundled helper, not a phantom.
• Installing the published version in a clean repo prompts for permission instead of proceeding silently, because the finding is critical or high.
• tessl install --accept-warnings installs the same plugin with no prompt, proof you can keep CI moving once a human has reviewed the finding.

If the install never prompts, the two usual causes are that the finding scored below high, or the workspace has the security review disabled. Confirm the registry page actually shows the finding before assuming the gate is broken.

What you keep

You walk away having seen the security review from both ends: a plugin you published with a real vulnerability and watched get flagged, and the install prompt you met as the consumer downstream. That's the half of skill quality the evals can't measure: not is this good? but is this safe to bring into my repo? Pair it with the review and task evals from the last three lessons and the continuous gate you're about to wire up next, and every skill your team publishes carries both signals, on every version, for everyone who installs it.