llm-evaluation

Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.

Quality

—

Does it follow best practices?

Impact

—

No eval scenarios have been run

Securityby

Passed

No known issues

Quality

Content

57%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is well-structured and correctly uses progressive disclosure with a real reference file, but it spends tokens re-explaining metrics Claude already knows, ships a code example that isn't fully runnable as written, and never articulates the evaluation workflow with validation checkpoints.

Suggestions

Trim the per-metric glosses (BLEU/ROUGE/Accuracy/etc.) to bare names or move them to references/details.md, since Claude already knows these definitions; cut the 'Master comprehensive...' marketing intro.

Make the Quick Start code copy-paste runnable by either defining the referenced helper functions (calculate_accuracy, calculate_bleu, calculate_bertscore) or clearly marking them as user-supplied stubs with a one-line note.

Add an explicit numbered evaluation workflow with a validation checkpoint, e.g. 1) define metrics, 2) build suite, 3) run on test cases, 4) check scores against a baseline and abort/review if any metric regresses.

Dimension	Reasoning	Score
Conciseness	Mostly efficient, but metric glosses like 'BLEU: N-gram overlap (translation)' and 'Accuracy: Percentage correct' re-explain concepts Claude already knows, and the 'Master comprehensive...' intro is marketing fluff that could be cut.	2 / 3
Actionability	The EvaluationSuite framework is real code, but it depends on undefined helpers (calculate_accuracy, calculate_bleu, check_groundedness) and your_model, and the per-metric sections describe metrics rather than giving executable computation guidance.	2 / 3
Workflow Clarity	The evaluation workflow (define metrics, build suite, run, analyze) is only implicit in the code example, with no explicit validation checkpoints or error-recovery feedback loop articulated.	2 / 3
Progressive Disclosure	A well-organized overview with a clearly signaled, one-level-deep reference to the real bundle file references/details.md, matching the top anchor for clear navigation.	3 / 3
	Total	9 / 12 Passed

Description

85%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, well-structured description that follows the prescribed what-plus-when pattern with an explicit trigger clause and concrete evaluation approaches. Its only weakness is trigger phrasing that leans slightly formal and omits colloquial variations a user might actually say.

Dimension	Reasoning	Score
Specificity	Names three concrete evaluation approaches — 'automated metrics, human feedback, and benchmarking' — matching the anchor for listing multiple specific concrete actions.	3 / 3
Completeness	Clearly answers both what ('Implement comprehensive evaluation strategies...') and when via an explicit 'Use when...' clause, matching the top anchor.	3 / 3
Trigger Term Quality	'testing LLM performance' and 'measuring AI application quality' are natural, but 'establishing evaluation frameworks' leans jargony and common colloquial variations (e.g. 'evaluate my model', 'benchmark my LLM') are absent.	2 / 3
Distinctiveness Conflict Risk	The LLM-evaluation niche is clear and the triggers are specific enough to be unlikely to fire for unrelated skills.	3 / 3
	Total	11 / 12 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: wshobson/agents
Commit: 5cc2549

Reviewed: 7 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.