CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

generate-evals

Generates and verifies eval scenarios for a repository plugin — covering its skills, commands, and hooks — using the tessl-compatible evals/ layout (task.md, criteria.json, scenario.json) so they run under `tessl eval run` and grade offline with any judge. Authors self-gradable scenarios, runs the offline verification ladder, and commits. This skill never restructures plugins. Use when the user asks to build evals for a plugin, generate eval scenarios, or verify that a skill actually works.

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A tightly written, well-sequenced operational workflow with a genuine verification ladder and highly specific, non-inferable rules for scenario design, fixtures, and shims. The main gaps are that the hardest artifacts (setup.sh, recording shims) are described rather than exemplified, and everything is inlined in one file with no bundle references.

DimensionReasoningScore

Conciseness

The body is dense with non-obvious, domain-specific rules (the tessl manifest recipe, the `scenario-N` downloader-collision rationale, shim flag-parsing like 'taking the token after a value-taking flag as that value even when it starts with `-`') and assumes competence, with no padding explaining concepts Claude already knows. It falls short of 5 only through minor redundancy: 'Never grade behaviour the unit does not document' appears in section 3.7 and again in Non-goals, and the HARD BOUNDARY overlaps the frontmatter's 'never restructures plugins'.

4 / 5

Actionability

Executable commands appear throughout ("jq '[.checklist[].max_score] | add' returns exactly 100", "git worktree add .worktrees/eval-<plugin> -b evals/<plugin> HEAD", "make test && make validate") plus exact JSON templates for scenario.json and criteria.json. It is not 5 because the two trickiest artifacts — inputs/setup.sh and the recording shims — are specified by description rather than shown as copy-paste code, and the worked example lives outside the skill bundle ('See the git-workflow PR scenario').

4 / 5

Workflow Clarity

Seven clearly sequenced phases (inventory, structure, design, offline gradeability, verification ladder, commit, batching) with an explicit checklist section headed 'run ALL before claiming done' containing exact commands and expected outcomes, plus explicit error-recovery and stop conditions ('If it fails for reasons unrelated to your scenarios, stop and report - do not modify manifests'). Batch and destructive operations (committing, temp-dir cleanup) are validation-gated, so the destructive/batch cap does not apply. Not 4: validation checkpoints are pervasive, not just present.

5 / 5

Progressive Disclosure

The body is well-sectioned (##/###) with clearly signaled, one-level-deep external pointers ('plugins/git-workflow/evals/ in this repository is the canonical worked example - read one of its scenarios before writing new ones') and no nested reference chains. It is not 5 because no bundle reference files exist (no references/, scripts/, or assets/), so dense detail such as the shim specification and the section 7 orchestrator batching notes is inlined in a ~220-line body that could be split; it is not 3 because the inlined material mostly belongs in the main workflow file and organization is good.

4 / 5

Total

17

/

20

Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it explicitly states what the skill does (generate, verify, author, run, commit) with concrete artifacts and file layout, and includes an explicit 'Use when...' clause with three natural trigger phrases. Minor deductions for one slightly broad trigger phrase and a few missing natural synonyms.

DimensionReasoningScore

Specificity

Multiple specific concrete actions with comprehensive coverage: 'Generates and verifies eval scenarios', 'Authors self-gradable scenarios, runs the offline verification ladder, and commits', anchored by concrete artifacts '(task.md, criteria.json, scenario.json)' and the runner '`tessl eval run`'. Not 4: the coverage spans generation, verification, grading mode, and commit with no minor gaps in the skill's scope.

5 / 5

Completeness

Explicitly answers both questions: what ('Generates and verifies eval scenarios for a repository plugin... Authors self-gradable scenarios, runs the offline verification ladder, and commits') and when ('Use when the user asks to build evals for a plugin, generate eval scenarios, or verify that a skill actually works') with three concrete trigger phrases. Matches the top anchor exactly.

5 / 5

Trigger Term Quality

Natural phrases users would say are present: 'build evals for a plugin', 'generate eval scenarios', 'verify that a skill actually works', plus 'evals', 'plugin', 'skill' as domain keywords. Not 5: common variations such as 'write evals', 'test scenarios', or grading-oriented phrasings are absent as explicit triggers; not 3: the included keywords are relevant and natural rather than missing common variations.

4 / 5

Distinctiveness Conflict Risk

A clear niche (tessl-compatible plugin evals) with an explicit boundary ('This skill never restructures plugins'), but the trigger 'verify that a skill actually works' is broad enough that a user wanting ad-hoc manual verification could invoke the skill wrongly — minor overlap risk with a closely related testing/verification skill. Not 5 for that reason; not 3 because the domain and other triggers are far more specific than generic.

4 / 5

Total

18

/

20

Passed

Validation

87%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

allowed_tools_field

'allowed-tools' contains unusual tool name(s)

Warning

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

14

/

16

Passed

Repository

rikdc/ai-skills

Path

plugins/eval-workflow/skills/generate-evals/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.