CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

eval-creator-ci

[Beta] CI-only eval regression runner using gh-aw (GitHub Agentic Workflows). Runs all eval cases in .evals/ on a schedule or per-PR, reports pass/fail results, and can block merges on regressions. Also creates new eval cases from promoted patterns flagged by learning-aggregator-ci. Use when: you want automated regression testing of promoted rules in CI/headless pipelines. For interactive eval creation and runs, use eval-creator.

66

Quality

83%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

SKILL.md
Quality
Evals
Security

Quality

Content

75%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-organized, largely actionable CI skill body with concrete commands, schemas, and a properly structured one-level-deep reference bundle. The main improvement opportunities are removing duplication between the body and references/workflow-example.md, moving the inline output schema to a reference, and fixing the workflow file extension.

Suggestions

Trim the run-mode verification-method definitions and trigger YAML duplicated from references/workflow-example.md to brief pointers, keeping the body lean and deferring detail to the reference (improves conciseness and progressive_disclosure).

Move the full eval_creator_ci output schema into a reference file (e.g., references/output-schema.md) and keep only a short example or key summary inline.

Correct the workflow install path to use a .yml/.yaml extension ('.github/workflows/eval-creator-ci.yml'), since GitHub Actions will not pick up a .md workflow file.

DimensionReasoningScore

Conciseness

The body is efficient and assumes competence: install commands, a CI contract, numbered mode workflows, and an output schema with no filler or explanations of concepts Claude already knows. It is not anchor 5 because the run-mode verification-method descriptions (grep-check, command-check, file-check, rule-check) duplicate what is already detailed in references/workflow-example.md and could be trimmed to a pointer.

4 / 5

Actionability

Mostly executable guidance: 'gh skill install pskoett/pskoett-skills eval-creator-ci', 'Validate: gh aw compile (add --actionlint --zizmor for security scan)', a concrete trigger YAML block, and a complete output schema. It falls short of anchor 5 because of small gaps, e.g., instructing users to copy the workflow into '.github/workflows/eval-creator-ci.md' where GitHub requires a .yml/.yaml extension.

4 / 5

Workflow Clarity

Both modes are given as ordered numbered sequences with checkpoints: precondition checks that mark cases 'skip', per-case pass/fail/skip recording, an explicit 'gh aw compile' validation step in the authoring workflow, and gate-policy decision logic in the workflow reference. It is not anchor 5 because no explicit fix-and-retry feedback loop is described when validation or a gate check fails.

4 / 5

Progressive Disclosure

Good structure overall: the body is a well-sectioned overview and correctly points to a real one-level-deep reference ('Copy references/workflow-example.md'), which exists and contains the full workflow templates. It is not anchor 5 because the 40-line output schema is inlined in SKILL.md rather than split out, and the body duplicates reference content (verification method definitions, trigger YAML) instead of deferring to it.

4 / 5

Total

16

/

20

Passed

Description

83%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description that states concrete capabilities, an explicit Use-when clause, and a hand-off to the interactive sibling skill. The main weaknesses are second-person phrasing in the trigger clause and trigger terms that cover the domain well but lack natural synonyms users would say.

Suggestions

Rewrite the trigger clause in third person, e.g., 'Use when running automated regression tests of promoted rules in CI/headless pipelines' instead of 'Use when: you want...'.

Add natural trigger variations users would say, such as 'run the evals', 'eval suite', or '.evals directory', to strengthen trigger_term_quality toward anchor 5.

DimensionReasoningScore

Specificity

Lists multiple concrete actions ('Runs all eval cases in .evals/ on a schedule or per-PR, reports pass/fail results, and can block merges on regressions. Also creates new eval cases from promoted patterns'), which matches the anchor-5 example. However, the rubric penalizes second-person phrasing, and the trigger clause 'Use when: you want automated regression testing' addresses the reader directly, reducing the score by 1.

4 / 5

Completeness

Explicitly answers both questions: the 'what' covers running, reporting, gating, and creating evals, and an explicit 'Use when: you want automated regression testing of promoted rules in CI/headless pipelines' clause states the 'when' with concrete trigger phrases. It also adds a hand-off clause ('For interactive eval creation and runs, use eval-creator'), which anchor 4's 'when could be more explicit' criticism does not apply to.

5 / 5

Trigger Term Quality

Good keyword coverage including 'eval regression', 'pass/fail', 'CI', 'headless pipelines', 'block merges', and 'gh-aw', matching the anchor-4 example ('PDF files, forms, document extraction'). It is not anchor 5 because common variations users would naturally say are missing (e.g., 'run the evals', 'eval suite', '.evals directory'), and it is clearly above anchor 3 since several natural domain terms are present.

4 / 5

Distinctiveness Conflict Risk

The CI/headless-only framing and the explicit disambiguation ('For interactive eval creation and runs, use eval-creator') establish a clear niche, but the skill sits in a family of closely related skills (eval-creator, learning-aggregator-ci) with overlapping vocabulary like 'creates new eval cases', matching anchor 4's 'minor overlap risk with closely related skills' rather than anchor 5's minimal conflict risk.

4 / 5

Total

17

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

pskoett/pskoett-ai-skills

Path

plugin/skills/eval-creator-ci/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.