agent-evaluation

You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in production. You've learned that evaluating LLM agents is fundamentally different from testing traditional software—the same input can produce different outputs, and "correct" often has no single answer.

Quality

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./plugins/antigravity-bundle-agent-architect/skills/agent-evaluation/SKILL.md

The canonical home for this skill is agent-evaluation in sickn33/agentic-awesome-skills

Quality

Content

14%Scale 1-3

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

This skill is essentially a skeleton or outline with no actionable content. It names concepts (statistical testing, behavioral contracts, adversarial testing) without providing any concrete guidance, code, commands, or detailed workflows. The Sharp Edges table contains placeholder comments instead of actual solutions, and the introductory text explains things Claude already knows without adding novel, project-specific knowledge.

Suggestions

Add concrete, executable code examples for each pattern (e.g., a Python function that runs an agent test N times and computes pass rate with confidence intervals for 'Statistical Test Evaluation').

Replace the Sharp Edges placeholder comments (e.g., '// Bridge benchmark and production evaluation') with actual step-by-step solutions or mitigation strategies.

Define a clear multi-step evaluation workflow with explicit validation checkpoints, such as: 1) Define behavioral contracts → 2) Write test cases → 3) Run N trials → 4) Analyze distributions → 5) Flag regressions if pass rate drops below threshold.

Either flesh out each pattern section with detailed guidance inline, or create separate reference files (e.g., STATISTICAL_TESTING.md, ADVERSARIAL_TESTING.md) and link to them from the main skill.

Dimension	Reasoning	Score
Conciseness	The introductory paragraphs explain concepts Claude already understands (what makes agent evaluation different from traditional testing). The tables and pattern/anti-pattern sections are reasonably concise but the overall content has filler without adding actionable value.	2 / 3
Actionability	There are no concrete code examples, commands, or executable guidance anywhere. Patterns like 'Statistical Test Evaluation' and 'Behavioral Contract Testing' are named but never explained with actual implementation steps, code, or specific techniques. The Sharp Edges table has comments instead of actual solutions.	1 / 3
Workflow Clarity	There is no workflow, sequence of steps, or process defined. The skill lists pattern names and anti-pattern names without describing how to actually perform agent evaluation. No validation checkpoints or feedback loops are present for what should be a multi-step evaluation process.	1 / 3
Progressive Disclosure	The content is a flat list of headings with minimal substance under each. There are no references to detailed files, no links to examples or deeper documentation, and the structure gives the appearance of organization without actual content to disclose progressively.	1 / 3
	Total	5 / 12 Passed

Description

0%Scale 1-3

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

This description reads as a persona/backstory rather than a functional skill description. It provides no concrete actions, no explicit trigger guidance, and no clear indication of when Claude should select this skill. The use of second-person voice ('You're a quality engineer') also violates the third-person convention.

Suggestions

Replace the persona narrative with concrete actions the skill performs, e.g., 'Designs evaluation frameworks for LLM agents, creates test suites, defines metrics, and builds scoring rubrics.'

Add an explicit 'Use when...' clause with natural trigger terms like 'evaluate agent', 'test LLM', 'evals', 'agent quality', 'accuracy metrics', 'benchmark'.

Rewrite in third person voice (e.g., 'Evaluates LLM agent performance...') instead of second person ('You're a quality engineer').

Dimension	Reasoning	Score
Specificity	The description uses vague, abstract language about being a 'quality engineer' and philosophical statements about LLM evaluation challenges. It names no concrete actions like 'create test suites', 'run evaluations', or 'measure accuracy'.	1 / 3
Completeness	The description fails to answer both 'what does this do' and 'when should Claude use it'. There is no 'Use when...' clause, no explicit triggers, and the 'what' is entirely absent — it only describes a persona and philosophy, not capabilities.	1 / 3
Trigger Term Quality	Contains technical jargon like 'benchmarks', 'production', and 'LLM agents' but lacks natural user trigger terms. A user needing this skill would likely say things like 'evaluate', 'test my agent', 'evals', 'accuracy', 'metrics' — none of which appear.	1 / 3
Distinctiveness Conflict Risk	The description is so vague about its actual function that it could overlap with any testing, QA, or AI-related skill. Without concrete actions or domain-specific triggers, it provides no clear niche to distinguish it from other skills.	1 / 3
	Total	4 / 12 Passed

Validation

90%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 10 / 11 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	10 / 11 Passed

Repository: administrakt0r/AI-Agents-Safe-Coding-Skills
Path: plugins/antigravity-bundle-agent-architect/skills/agent-evaluation/SKILL.md
Commit: 7520423

Reviewed: ~11 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.