eval-harness

Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles. Use when a Claude Code workflow needs a formal eval before it is trusted or changed.

Quality

57%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Medium

Suggest reviewing before use

Fix and improve this skill with Tessl

tessl review fix ./skills/eval-harness/SKILL.md

Quality

Content

57%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is well-structured and actionable with strong templates and executable grader examples, but it repeats grader/pass@k material across sections and lacks validation feedback loops and any progressive disclosure via separate reference files.

Suggestions

Add an explicit validation loop in the Evaluate step: 'If any eval fails, fix the change and re-run until all pass before reporting.'

De-duplicate the grader types and pass@k guidance that appear both in the main sections and the 'Product Evals (v1.8)' section.

Move the full 'Example: Adding Authentication' walkthrough and Product Evals detail into separate reference files linked from SKILL.md.

Dimension	Reasoning	Score
Conciseness	Mostly efficient with concrete templates, but the 'Product Evals (v1.8)' section restates grader types and pass@k guidance already covered earlier, and the time-sensitive '(v1.8)' tag adds version noise that could be trimmed.	3 / 5
Actionability	Provides concrete, mostly copy-paste-ready templates (capability/regression eval formats, grader prompts, report layout) and executable bash checks (grep, npm test), with only minor gaps where steps are placeholders like '[Run each capability eval]'.	4 / 5
Workflow Clarity	The Define → Implement → Evaluate → Report sequence is clear, but the Evaluate step lacks an explicit validation feedback loop (fix and re-run on failure); per the rubric, batch/regression operations without validation checkpoints cap workflow clarity at 3.	3 / 5
Progressive Disclosure	Good section headers structure the content, but everything is inlined in a single ~270-line SKILL.md with no bundle files or one-level-deep references; substantial material (full auth example, Product Evals detail) that could live in separate files is inlined.	3 / 5
	Total	13 / 20 Passed

Description

57%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description clearly states what the skill does and when to use it, with an explicit trigger clause and a recognizable niche. It is held back by generic capability language and missing common trigger synonyms like 'benchmark' or 'regression'.

Suggestions

Replace abstract phrasing with concrete actions, e.g. 'Define pass/fail criteria, run pass@k metrics, and generate eval reports for Claude Code workflows.'

Add natural trigger terms users would actually say, such as 'benchmarking', 'regression tests', or 'eval-driven development'.

Make the 'when' clause more specific, e.g. 'Use when adding prompt or agent changes that need regression coverage before release.'

Dimension	Reasoning	Score
Specificity	Names the domain ('Formal evaluation framework', 'eval-driven development (EDD) principles') but concrete actions are minimal and generic — no specific actions like defining pass/fail criteria, running pass@k metrics, or generating reports are stated.	2 / 5
Completeness	Has both a 'what' ('Formal evaluation framework for Claude Code sessions implementing eval-driven development') and an explicit 'when' ('Use when a Claude Code workflow needs a formal eval before it is trusted or changed'), though the 'when' could be more specific about concrete trigger scenarios.	4 / 5
Trigger Term Quality	Includes some relevant natural terms ('formal eval', 'Claude Code workflow', 'trusted or changed') but misses common variations and synonyms users would say such as 'benchmark', 'regression tests', or 'pass@k'.	3 / 5
Distinctiveness Conflict Risk	Targets a fairly distinct niche (formal evals / EDD for Claude Code workflows) with minimal overlap risk; it is mostly distinguishable from general testing or coding skills.	4 / 5
	Total	13 / 20 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
metadata_version	'metadata.version' is missing	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	14 / 16 Passed

Repository: affaan-m/ECC
Path: skills/eval-harness/SKILL.md
Commit: d8409a4

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.