eval-harness

Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles

Quality

57%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Medium

Suggest reviewing before use

Fix and improve this skill with Tessl

tessl review fix ./skills/eval-harness/SKILL.md

The canonical home for this skill is eval-harness in affaan-m/ECC

Quality

Content

65%Weight 40%Scale 1-3

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable with concrete graders and commands and a clear four-phase workflow, but it is verbose with redundant sections and lacks validation feedback loops and any external-reference structure.

Suggestions

Collapse the duplicated 'Product Evals (v1.8)' section into the earlier Grader Types / pass@k sections to remove redundancy and cut length.

Add explicit validation checkpoints in the Evaluate step (e.g. 'if a capability eval fails, fix and re-run before reporting') to turn the placeholder into a real feedback loop.

Move the full 'Example: Adding Authentication' and detailed grader prompts into a references file (e.g. references/eval-templates.md) and signal them from the body to improve progressive disclosure.

Dimension	Reasoning	Score
Conciseness	The 271-line body restates concepts Claude already knows ('evals as the unit tests of AI development', pass@k definitions) and duplicates material — 'Grader Types' and pass@k guidance appear both early and again in the 'Product Evals (v1.8)' section — so it is mostly efficient but padded and could be tightened, matching anchor 2.	2 / 3
Actionability	It provides copy-paste-ready executable graders (grep/npm test/bash snippets), concrete slash commands (/eval define, /eval check, /eval report), and explicit file paths (.claude/evals/<feature>.md), matching the fully-executable anchor 3.	3 / 3
Workflow Clarity	A clear Define→Implement→Evaluate→Report sequence exists, but the Evaluate step is a placeholder ('[Run each capability eval, record PASS/FAIL]') and there are no explicit validation/feedback checkpoints for release-gated regression workflows, so per the guidelines workflow clarity is capped at 2.	2 / 3
Progressive Disclosure	The skill is a single 271-line SKILL.md with no bundle files; it is well-sectioned but exceeds the under-50-line simple-skill allowance and keeps content (full worked example, grader details) inline rather than split into signaled one-level references, matching anchor 2.	2 / 3
	Total	9 / 12 Passed

Description

50%Weight 40%Scale 1-3

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description clearly identifies the domain (eval-driven development for Claude Code) but lacks concrete capability actions, natural trigger terms, and any explicit 'when to use' guidance, leaving it competent but incomplete.

Suggestions

Append a 'Use when...' clause naming concrete triggers users would say, e.g. 'Use when setting up evals, defining pass/fail criteria, or benchmarking Claude Code agent reliability'.

List concrete actions in the description (define pass/fail criteria, measure pass@k, build regression suites, generate eval reports) to raise specificity above the domain-only level.

Add natural-language variations (evals, pass@k, regression tests, benchmarking) so trigger matching works on the terms users actually say.

Dimension	Reasoning	Score
Specificity	The phrase 'Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles' names the domain clearly but lists no concrete actions (e.g. define pass/fail criteria, measure pass@k), matching the 'names domain and some actions' anchor and falling short of the multiple-concrete-actions anchor 3.	2 / 3
Completeness	It states what the skill does but provides no 'Use when...' clause or equivalent trigger guidance, so per the judging guidelines completeness is capped at 2 (has what, when is missing).	2 / 3
Trigger Term Quality	Terms like 'evaluation framework' and 'eval-driven development (EDD)' are relevant but jargon-heavy, and the description omits common natural variations a user would say ('write evals', 'regression tests', 'benchmark my agent'); it is above the no-natural-keywords anchor but below the good-coverage anchor.	2 / 3
Distinctiveness Conflict Risk	The EDD-for-Claude-Code niche is fairly specific, but 'evaluation framework' is generic enough to overlap with general testing/review skills and no distinct triggers are given, matching the 'somewhat specific but could still overlap' anchor.	2 / 3
	Total	8 / 12 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
metadata_version	'metadata.version' is missing	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	14 / 16 Passed

Repository: ysyecust/everything-claude-code
Path: skills/eval-harness/SKILL.md
Commit: a9cbecb

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.