CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

eval-creator

[Beta] Creates permanent eval cases from promoted learnings and runs regression checks against them. Turns failures into test cases that prevent silent regression. This is the outer loop's regress-test step. Use when a learning is promoted and has a clear pass/fail condition, or on cadence to verify promoted rules still hold.

Quality

81%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

The canonical home for this skill is eval-creator in pskoett/pskoett-ai-skills

SKILL.md
Quality
Evals
Security

Quality

Content

70%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is a well-sequenced, genuinely actionable instruction skill — the execution checklist, evidence-strength gate, and recovery feedback loops are exemplary for a batch verification workflow. Its weaknesses are repetition (blog quote, duplicated verification-method definitions) and the complete absence of progressive disclosure: everything, including secondary material like skill-validation details, lives in one long SKILL.md.

Suggestions

Move the full eval case template and the Skill Validation (skill-check) details into references/ files (e.g. references/eval-case-template.md, references/skill-validation.md), keeping only a compact example plus a one-line pointer in SKILL.md.

Delete the blog quote and the intro paragraph that repeats the description, and consolidate the verification-method definitions so they appear once (in Execution) rather than twice.

Make "Run by Pattern-Key" and "Run by Area" concrete by specifying the actual filtering mechanism (e.g. the grep pattern or glob used against .evals/cases/).

DimensionReasoningScore

Conciseness

The body is mostly dense and functional (templates, checklists, commands), but includes unnecessary padding: the motivational blog quote ("If a failure taught you something important..."), an intro paragraph that repeats the frontmatter description verbatim, and the verification methods being defined twice — once in the Eval Case Format section and again under Execution. Not 2 because there is no basic-concept exposition and most sections earn their tokens.

3 / 5

Actionability

Highly actionable throughout: a complete eval-case markdown template, five verification methods with concrete examples (grep patterns, "npm run typecheck", "python .claude/skills/skill-creator/scripts/quick_validate.py <target>"), a runnable mcp-script example, and report/index formats. Minor gaps: "Run by Pattern-Key — Filter to evals matching a specific pattern" and "Run by Area" give no concrete filtering mechanics.

4 / 5

Workflow Clarity

The Execution section is a clear 6-step sequence with explicit validation checkpoints (precondition check → skip, evidence-strength check that marks "fail" with "insufficient_evidence" rather than upgrading claims), the eval case template mandates a Recovery Action ending in "Re-run this eval to verify" (a feedback loop), and the regression report format closes the loop. This matches the anchor-5 pattern of explicit validation steps plus error-recovery feedback loops for a batch operation.

5 / 5

Progressive Disclosure

Sections are clearly headed and well-ordered, but this is a ~290-line single-file skill with zero bundle files: the ~90-line eval case template, the ~50-line Skill Validation section, and the mcp-scripts extension section are all inlined when they clearly belong in one-level-deep reference files. Not 4 because content that should be separate is inline throughout, not just in minor spots.

3 / 5

Total

15

/

20

Passed

Description

83%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description that clearly states what the skill does and gives explicit, multi-condition trigger guidance in third person. Its main weaknesses are mild redundancy between the first two sentences and reliance on ecosystem-internal jargon ("promoted learnings", "outer loop") that limits natural trigger matching outside its own workflow.

Suggestions

Replace the redundant second sentence ("Turns failures into test cases that prevent silent regression") with a distinct capability, e.g. generating the regression report or updating the eval index.

Add one or two natural user-facing synonyms such as "regression test" or "test case" alongside the ecosystem terms "promoted learning" and "outer loop".

DimensionReasoningScore

Specificity

Lists several concrete actions — "Creates permanent eval cases from promoted learnings", "runs regression checks", "verify promoted rules still hold" — but "Creates permanent eval cases" and "Turns failures into test cases" restate the same action, leaving coverage with minor gaps rather than comprehensive distinct actions.

4 / 5

Completeness

Explicitly answers both: what ("Creates permanent eval cases from promoted learnings and runs regression checks") and when ("Use when a learning is promoted and has a clear pass/fail condition, or on cadence to verify promoted rules still hold") — the when-clause is concrete and multi-condition, matching the anchor-5 example's structure. Not 4 because the trigger guidance is more specific than the anchor-4 example's generic 'Use when working with X'.

5 / 5

Trigger Term Quality

Good natural keywords like "eval", "regression checks", "pass/fail condition", and "promoted rules", but terms like "promoted learning", "outer loop", and "regress-test step" are workflow-internal jargon users outside this ecosystem would not naturally say, and common variations (e.g. "test case", "regression test") are missing.

4 / 5

Distinctiveness Conflict Risk

Clear niche (eval cases derived from promoted learnings, regression verification of the harness) that is well-distinguished from sibling skills like learning-aggregator and self-improvement, but "runs regression checks" carries minor overlap risk with generic test-runner skills. Not 5 because the niche is defined by ecosystem jargon rather than fully distinct trigger phrases.

4 / 5

Total

17

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

pskoett/pskoett-ai-skills

Path

agent-plugin/skills/eval-creator/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.