llm-evaluation

Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.

Quality

76%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./plugins/llm-application-dev/skills/llm-evaluation/SKILL.md

Quality

Content

65%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is well-structured with a strong, executable Quick Start and clean progressive disclosure to a real reference file. It is held back by an unnecessary metric glossary and the absence of an explicit end-to-end workflow with checkpoints.

Suggestions

Trim or relocate the metric glossary (BLEU/ROUGE/METEOR/etc. one-liners) into references/details.md, since Claude already knows these definitions; keep only the category structure in the overview.

Add a short numbered workflow for running an evaluation (e.g., 1. select metrics, 2. assemble test cases, 3. run EvaluationSuite, 4. inspect raw_scores and aggregate, 5. re-run on regressions) so the process is explicit rather than implied by code.

Either inline minimal definitions for the helper functions referenced in the Quick Start (calculate_accuracy, calculate_bleu, calculate_bertscore, check_groundedness) or explicitly note they are defined in references/details.md, so the example is self-contained.

Dimension	Reasoning	Score
Conciseness	The Quick Start code is dense and earns its place, but the metric glossary ('BLEU: N-gram overlap (translation)', 'ROUGE: Recall-oriented (summarization)', etc.) explains concepts Claude already knows and could be trimmed or moved to the reference file.	3 / 5
Actionability	The Quick Start provides a complete, copy-paste-ready EvaluationSuite framework with a usage example, but the referenced helper functions (calculate_bleu, calculate_bertscore, check_groundedness) are not defined in the body, leaving a minor gap filled only by the external reference.	4 / 5
Workflow Clarity	A rough sequence is implied by the Quick Start code (define metrics, build suite, run evaluate, read results), but there is no explicit numbered workflow or validation/checkpoint guidance for running an evaluation end-to-end.	3 / 5
Progressive Disclosure	The body is a well-organized overview that pushes detailed patterns to a single one-level-deep reference ('references/details.md'), which exists as a real file, with a clearly signaled pointer and easy navigation.	5 / 5
	Total	15 / 20 Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is well-constructed: third-person voice, explicit what-and-when structure, and domain-specific triggers. Its only weakness is moderate specificity, grouping capabilities into three broad categories rather than listing concrete actions.

Dimension	Reasoning	Score
Specificity	Names the domain and several concrete approaches ('automated metrics, human feedback, and benchmarking'), but groups them into three broad categories rather than enumerating many specific actions, so it stops short of comprehensive coverage.	4 / 5
Completeness	Explicitly answers both what ('Implement comprehensive evaluation strategies...using automated metrics, human feedback, and benchmarking') and when ('Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks') with concrete trigger phrases.	5 / 5
Trigger Term Quality	Includes natural phrases users would say ('testing LLM performance, measuring AI application quality, establishing evaluation frameworks'); coverage is good but misses common synonyms like 'evals' or standalone 'benchmarks'.	4 / 5
Distinctiveness Conflict Risk	Occupies a clear niche (LLM evaluation) with distinct triggers tied to eval/testing terminology, creating minimal overlap risk with other skills.	5 / 5
	Total	18 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: wshobson/agents
Path: plugins/llm-application-dev/skills/llm-evaluation/SKILL.md
Commit: d6837ae

Reviewed: about 12 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.