CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

eval-creator

[Beta] Creates permanent eval cases from promoted learnings and runs regression checks against them. Turns failures into test cases that prevent silent regression. This is the outer loop's regress-test step. Use when a learning is promoted and has a clear pass/fail condition, or on cadence to verify promoted rules still hold.

68

Quality

85%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

The canonical home for this skill is eval-creator in pskoett/pskoett-ai-skills

SKILL.md
Quality
Evals
Security

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is a strong, actionable spec with concrete templates, executable commands, and a well-sequenced workflow including validation checkpoints and recovery loops; its only weak spots are minor redundancy in the opening and some long inline sections that could be extracted into reference files.

DimensionReasoningScore

Conciseness

The body is dense and task-specific throughout, with only minor instances of over-explanation that could be trimmed — the opening restates the frontmatter description and the blog quote ('If a failure taught you something important…') is motivational padding.

4 / 5

Actionability

Provides fully usable, copy-paste-ready guidance: a complete eval case frontmatter template, concrete verification-method field schemas (target/pattern/expect, command/expect_exit), an executable `python .claude/skills/skill-creator/scripts/quick_validate.py <target>` invocation, and a runnable mcp-script code example.

5 / 5

Workflow Clarity

The Execution section lays out a clear 6-step sequence with explicit validation checkpoints (precondition check → skip, evidence-strength check → fail with insufficient_evidence) and a recovery feedback loop (diagnose → fix → re-run to verify).

5 / 5

Progressive Disclosure

Well-organized with clear section headers and self-contained templates needing no external references; the skill-validation and custom-verification-methods content is somewhat lengthy inline and could split into reference files, but placement is reasonable.

4 / 5

Total

18

/

20

Passed

Description

76%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description clearly conveys both capability and trigger conditions with concrete language, but its trigger terms are couched in the outer-loop system's internal jargon rather than natural user phrasing, slightly limiting when-to-use recognizability.

Suggestions

Add plain-language trigger phrases users would actually say (e.g. 'Use when checking that previously fixed regressions haven't returned' or 'when verifying promoted rules still hold after a change') alongside the existing jargon.

Include natural synonyms/variations such as 'regression test', 'prevent repeat failures', and 'verify fixes still hold' to broaden trigger coverage.

Consider dropping or glossing the '[Beta]' tag and 'outer loop's regress-test step' phrasing in the description, since these assume context outsiders may not share.

DimensionReasoningScore

Specificity

Names several concrete actions ('Creates permanent eval cases from promoted learnings', 'runs regression checks against them', 'Turns failures into test cases that prevent silent regression'), though two overlap conceptually so coverage is strong but not fully comprehensive.

4 / 5

Completeness

Explicitly answers both what ('Creates permanent eval cases…runs regression checks') and when ('Use when a learning is promoted and has a clear pass/fail condition, or on cadence to verify promoted rules still hold') with concrete trigger phrases.

5 / 5

Trigger Term Quality

Relevant trigger terms exist ('promoted', 'clear pass/fail condition', 'regression', 'on cadence') but they lean on the system's own jargon ('promoted learnings', 'outer loop's regress-test step') rather than natural phrasings a user would spontaneously say, and common synonyms are missing.

3 / 5

Distinctiveness Conflict Risk

The 'outer loop's regress-test step' framing carves a clear niche distinct from general testing skills, with only minor overlap risk against generic test/regression skills.

4 / 5

Total

16

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

pskoett/pskoett-ai-skills

Path

agent-plugin/skills/eval-creator/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.