CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

eval-creator

[Beta] Creates permanent eval cases from promoted learnings and runs regression checks against them. Turns failures into test cases that prevent silent regression. This is the outer loop's regress-test step. Use when a learning is promoted and has a clear pass/fail condition, or on cadence to verify promoted rules still hold.

Quality

85%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

The canonical home for this skill is eval-creator in pskoett/pskoett-ai-skills

SKILL.md
Quality
Evals
Security

Quality

Content

77%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

Highly actionable and operationally precise: complete templates, concrete verification methods, and an execution loop with real validation checkpoints and feedback loops. The weaknesses are token efficiency (quote padding and duplicated method explanations) and the absence of progressive disclosure — a long single file with advanced material inlined that should live in reference files.

Suggestions

Split the Skill Validation section (quick_validate.py / run_eval.py details and when-to-create guidance) and the Custom Verification Methods (mcp-scripts) section into separate reference files, keeping one-line pointers with a brief inline example in SKILL.md.

Delete the motivational blog quote and merge the Execution section's tool mappings into the verification-method definitions in Eval Case Format so each method is explained once.

Add a brief 'Create an eval' step sequence (write the case file to .evals/cases/eval-YYYYMMDD-NNN.md, then add its row to EVAL_INDEX.md) so index updates on creation are as explicit as they are on runs.

DimensionReasoningScore

Conciseness

Mostly efficient, but it could be meaningfully tightened: the motivational blog quote ("If a failure taught you something important, it should become a permanent test case...") is padding, the verification methods are explained twice ("Eval Case Format" syntax blocks plus trivial tool mappings in "Execution" like "Use Grep tool to search target files"), and the ~50-line Skill Validation section re-documents skill-creator's validators in detail. This fits anchor 3 rather than anchor 4, where over-explanation would be only minor.

3 / 5

Actionability

Fully executable guidance throughout: a copy-paste-ready eval case template with concrete frontmatter fields, concrete verification blocks (grep target/pattern/expect, command/expect_exit, behavior-check with positive and negative controls and a real example command), an executable invocation ("python .claude/skills/skill-creator/scripts/quick_validate.py <target>"), a runnable mcp-script example, and templated report and index formats. Specific examples cover the common create/run/report cases, matching anchor 5; the only soft spots (one-line "Run by Pattern-Key"/"Run by Area") are minor variants of the fully specified run loop.

5 / 5

Workflow Clarity

The execution workflow is a numbered checklist with explicit validation checkpoints: precondition check (mark skip), comparison to expected result, and an evidence-strength gate that refuses to upgrade presence/structural evidence to behavioral claims. Recovery Action templates a feedback loop (diagnose, fix, re-run to verify) and failures flow into a structured report. This matches anchor 5 (explicit validation steps, error-recovery feedback loops, checklists); anchor 4's 'minor validation gaps' is not the case here.

5 / 5

Progressive Disclosure

The file is well-sectioned with clear headers, but it is a ~290-line monolith with no reference files at all, and specialized content that clearly belongs in separate files is inlined — notably the ~50-line Skill Validation section on skill-creator's validators and the Custom Verification Methods (mcp-scripts) extension. This matches anchor 3 (some structure, content that should be separate is inline); the skill is far beyond the under-50-lines exception that would let structure alone score 5.

3 / 5

Total

16

/

20

Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: third-person, concise, concrete about what it does, and explicit about when to use it with genuine trigger phrases. It occupies a distinct niche (regression testing of promoted learnings) with minimal conflict risk.

DimensionReasoningScore

Specificity

"Creates permanent eval cases from promoted learnings", "runs regression checks against them", and "Turns failures into test cases that prevent silent regression" list several concrete actions. It is not anchor 5 because coverage has minor gaps (e.g., listing/inspecting evals and the regression report output are not named), and not 3 because more than 1-2 specific actions are given.

4 / 5

Completeness

The what is explicit ("Creates permanent eval cases from promoted learnings and runs regression checks against them") and the when is explicit with concrete trigger phrases ("Use when a learning is promoted and has a clear pass/fail condition, or on cadence to verify promoted rules still hold"), matching anchor 5. Anchor 4 would require the when to be less explicit or specific, which is not the case.

5 / 5

Trigger Term Quality

Natural phrases a user of this workflow would say appear: "a learning is promoted", "clear pass/fail condition", "on cadence", "verify promoted rules still hold". A few natural variations (e.g., "eval case", "permanent test case") are present only obliquely, so it fits anchor 4 (good coverage, a few terms missing) rather than anchor 5's comprehensive synonym coverage.

4 / 5

Distinctiveness Conflict Risk

A clear niche (promoted learnings turned into permanent regression evals, "the outer loop's regress-test step") with distinct triggers tied to promotion and pass/fail conditions, minimizing overlap with adjacent skills like self-improvement. It does not fit anchor 4 because there is no notable overlap risk with closely related skills within the described ecosystem.

5 / 5

Total

18

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository

pskoett/pskoett-ai-skills

Path

plugin/skills/eval-creator/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.