CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

eval-creator-ci

[Beta] CI-only eval regression runner using gh-aw (GitHub Agentic Workflows). Runs all eval cases in .evals/ on a schedule or per-PR, reports pass/fail results, and can block merges on regressions. Also creates new eval cases from promoted patterns flagged by learning-aggregator-ci. Use when: you want automated regression testing of promoted rules in CI/headless pipelines. For interactive eval creation and runs, use eval-creator.

Quality

87%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

75%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured CI skill body: concrete commands, clearly sequenced run/create modes with precondition and gate checkpoints, a real and appropriately scoped reference file, and a defined output contract. The main gaps are the never-shown eval-case file format in create mode, some redundancy with the description, and minor content that belongs in the reference rather than the body.

Suggestions

Show a minimal concrete eval-case example (frontmatter + verification method) in create mode, or point to one in the reference file, so 'create the eval case file with proper frontmatter' is fully executable.

Trim the 'Differences from Interactive Version' table to the few rows not already implied by the description and Purpose sections, and move the pinned gh-aw version into the prerequisites line or a deprecation-friendly spot.

Move the full Trigger Configuration YAML into references/workflow-example.md and keep only the trigger-policy recommendation in the body to reduce duplication with the reference.

DimensionReasoningScore

Conciseness

The body is efficient and assumes competence — no re-teaching of known concepts — but the "Differences from Interactive Version" table partially duplicates the description and Purpose sections, and the pinned "v0.40.1+" is time-sensitive detail outside any deprecation section. Matches the 4 anchor (efficient, minor trimmable instances); not 5 because of that redundancy and version pin, not 3 because padding is incidental rather than a pattern.

4 / 5

Actionability

Mostly executable guidance: concrete install commands ("gh skill install pskoett/pskoett-skills eval-creator-ci"), validation command ("gh aw compile" with "--actionlint --zizmor"), concrete verification-method semantics, and a full output YAML schema. Falls short of 5 because create mode instructs "Create the eval case file in `.evals/cases/` with proper frontmatter" without ever showing the eval-case frontmatter format — a missing key detail for copy-paste readiness.

4 / 5

Workflow Clarity

Both modes are clearly sequenced with checkpoints: precondition checks with skip handling, expected-vs-actual comparison per verification method, gate-policy failure behavior, and a compile-validation step in the authoring workflow. Not 5 because there is no in-run error-recovery/feedback loop — failures are escalated to humans/self-improvement-ci (a defensible design choice, but the anchor requires explicit feedback loops).

4 / 5

Progressive Disclosure

Well-organized sections with one clearly signaled, verified, one-level-deep reference ("Copy `references/workflow-example.md` into `.github/workflows/eval-creator-ci.md`") that appropriately carries the executable workflow detail. Not 5 because the inline Trigger Configuration YAML block duplicates detail that belongs in the reference, and the ~215-line body carries the full output schema and data-flow diagram inline rather than splitting them out.

4 / 5

Total

16

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: third-person voice, concrete multi-action capability list, an explicit 'Use when' trigger clause, and deliberate disambiguation from the interactive sibling skill. Its only weakness is a mild reliance on skill-suite jargon over the most natural user phrasings in the trigger terms.

DimensionReasoningScore

Specificity

Lists multiple specific concrete actions with comprehensive coverage: "Runs all eval cases in .evals/ on a schedule or per-PR, reports pass/fail results, and can block merges on regressions. Also creates new eval cases from promoted patterns flagged by learning-aggregator-ci." This matches the 5 anchor (multiple concrete actions, comprehensive) rather than 4, since run, report, gate, and create modes all covered; not below 4 because there are no meaningful coverage gaps.

5 / 5

Completeness

Explicitly answers both: what ("Runs all eval cases... reports pass/fail results, and can block merges... Also creates new eval cases") and when ("Use when: you want automated regression testing of promoted rules in CI/headless pipelines"). This matches the 5 anchor — clear what AND when with a concrete trigger phrase.

5 / 5

Trigger Term Quality

Good natural-term coverage: "automated regression testing", "CI/headless pipelines", "regressions", "block merges", "per-PR", and the ".evals/" path. It falls short of the 5 anchor because it leans on suite-internal jargon ("promoted rules", "learning-aggregator-ci") and misses common variations a user might say ("eval suite", "quality gate"); it is above 3 because keywords are relevant and include several natural phrasings and a path extension.

4 / 5

Distinctiveness Conflict Risk

"[Beta] CI-only" pins a clear niche and the routing sentence "For interactive eval creation and runs, use eval-creator" explicitly defuses the one real overlap risk with the sibling interactive skill. Minimal conflict risk, matching the 5 anchor.

5 / 5

Total

19

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

pskoett/pskoett-ai-skills

Path

skills/eval-creator-ci/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.