evaluation

Build evaluation frameworks for agent systems. Use when testing agent performance systematically, validating context engineering choices, or measuring improvements over time.

Quality

44%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./skills/evaluation/SKILL.md

Quality

Content

31%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

This skill reads more like a conceptual overview or textbook chapter on agent evaluation than an actionable skill for Claude. It extensively explains concepts Claude already understands (non-determinism, multi-dimensional quality, etc.) while providing almost no executable code or concrete implementation guidance. The structure is disorganized with duplicate sections and references to non-existent bundle files.

Suggestions

Replace pseudocode examples with fully executable evaluation code — implement actual `assess_dimension()` logic using LLM-as-judge with a concrete prompt template, and provide a working evaluation runner script.

Cut the 'Evaluation Challenges' and 'Core Concepts' sections down to 2-3 bullet points each — Claude already understands non-determinism, multi-dimensional quality, and context-dependent failures.

Remove duplicate sections ('When to Use' appears twice, 'References' appears twice) and consolidate the guidelines/practical guidance sections which largely repeat each other.

Either create bundle files for the referenced materials (metrics reference, rubric templates, example test sets) or remove the references — currently they point to nothing and create false expectations.

Dimension	Reasoning	Score
Conciseness	Extensively verbose with many sections that explain concepts Claude already knows (what non-determinism is, why agents take different paths, what factual accuracy means). The 'Core Concepts' and 'Evaluation Challenges' sections are largely conceptual padding. The BrowseComp table is interesting but the surrounding explanation is over-elaborated. Multiple redundant sections (e.g., 'When to Use' appears twice, 'References' appears twice, guidelines repeat earlier content).	2 / 5
Actionability	The code examples are pseudocode-level at best — `assess_dimension()`, `load_rubric()`, and `weighted_average()` are undefined placeholder functions that cannot be executed. The test set example is a data structure but provides no actual evaluation logic. Most guidance is abstract ('Build evaluation pipelines that run automatically', 'Create rubrics with clear, actionable level descriptions') without concrete implementation details, specific commands, or copy-paste ready code.	2 / 5
Workflow Clarity	The 'Building Evaluation Frameworks' section provides an 8-step sequence, but steps are high-level and lack validation checkpoints. There's no feedback loop for when evaluations fail or produce unexpected results. For a skill involving systematic testing and pipeline building, the absence of explicit validation steps and error recovery is a notable gap.	3 / 5
Progressive Disclosure	No bundle files are provided despite multiple references to other skills and a 'Metrics Reference' document. The content is a monolithic wall of text (~300+ lines) that would benefit greatly from splitting into separate files (rubric templates, example implementations, metrics reference). References to internal skills are listed but point to nothing concrete. The duplicate 'References' and 'When to Use' sections suggest poor organization.	2 / 5
	Total	9 / 20 Passed

Description

57%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description establishes a clear niche (agent evaluation) and includes an explicit 'Use when' clause with reasonable trigger scenarios. However, it lacks specificity in what concrete actions the skill performs—'build evaluation frameworks' is a single vague action that doesn't convey the actual capabilities (e.g., creating test suites, defining metrics, running benchmarks). The trigger terms are adequate but miss common synonyms users might naturally use.

Suggestions

Add specific concrete actions like 'create test suites, define scoring rubrics, run benchmarks, compare agent configurations, generate performance reports' to replace the vague 'build evaluation frameworks'.

Include natural trigger term variations such as 'evals', 'benchmarks', 'metrics', 'accuracy', 'regression testing', or 'scoring' that users commonly use when discussing agent evaluation.

Dimension	Reasoning	Score
Specificity	The description names the domain ('evaluation frameworks for agent systems') but the actions are minimal and generic. 'Build evaluation frameworks' is a single high-level action without concrete sub-actions like 'create test cases', 'generate scoring rubrics', 'run benchmark suites', or 'produce comparison reports'.	2 / 5
Completeness	Has both 'what' (build evaluation frameworks for agent systems) and 'when' (testing agent performance, validating context engineering choices, measuring improvements over time). The 'when' clause is present and reasonably specific, though the 'what' could be more detailed about concrete deliverables.	4 / 5
Trigger Term Quality	Includes some relevant keywords like 'agent', 'evaluation', 'testing', 'performance', 'context engineering', and 'measuring improvements', but misses natural variations users might say such as 'benchmark', 'evals', 'accuracy', 'metrics', 'scoring', 'regression testing', or 'A/B testing'.	3 / 5
Distinctiveness Conflict Risk	The combination of 'evaluation frameworks' + 'agent systems' + 'context engineering' creates a fairly distinct niche. There's minor overlap risk with general testing/QA skills or generic agent-building skills, but the specific focus on agent evaluation and context engineering choices makes it mostly distinguishable.	4 / 5
	Total	13 / 20 Passed

Validation

90%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 10 / 11 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	10 / 11 Passed

Repository: administrakt0r/AI-Agents-Safe-Coding-Skills
Path: skills/evaluation/SKILL.md
Commit: 57c135b

Reviewed: 3 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.