agent-evaluation

You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in production. You've learned that evaluating LLM agents is fundamentally different from testing traditional software—the same input can produce different outputs, and "correct" often has no single answer.

Quality

28%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./plugins/AI-Agents-Safe-Coding-Skills/skills/agent-evaluation/SKILL.md

The canonical home for this skill is agent-evaluation in sickn33/agentic-awesome-skills

Quality

Content

26%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is section-organized but content-hollow: it offers abstract pattern names with no executable guidance, empty anti-pattern entries, stub solutions, a truncated sentence, and a meaningless 'When to Use' placeholder. It describes evaluation approaches rather than instructing how to perform them.

Suggestions

Replace abstract pattern one-liners with concrete executable guidance, e.g. a snippet that runs N trials of an agent task and aggregates the pass-rate distribution.

Add a sequenced evaluation workflow with validation checkpoints (define behavioral invariants → run N trials → aggregate distributions → flag flakiness/regression → re-validate after fixes).

Delete the verbose opening narrative and fix the truncated sentence; fill in the empty Anti-Pattern sections and replace the '//'-prefixed stubs in the Sharp Edges table with real remediation steps.

Dimension	Reasoning	Score
Conciseness	The opening two paragraphs are unnecessary motivational explanation of concepts Claude already knows ('evaluating LLM agents is fundamentally different from testing traditional software') and even truncate mid-sentence ('the goal isn't 100% test pass rate—it'), fitting 'Noticeably verbose; several unnecessary explanations or padded sections'.	2 / 5
Actionability	Patterns are abstract one-liners ('Run tests multiple times and analyze result distributions'), Anti-Patterns are empty headers, and Sharp Edges 'solutions' are stub comments ('// Bridge benchmark and production evaluation') — high-level hints with no specific executable steps, matching 'Minimal concrete guidance; high-level hints but missing the specific steps to execute'.	2 / 5
Workflow Clarity	There is no sequenced evaluation workflow at all — only categorized lists (Patterns, Anti-Patterns, Sharp Edges) with no ordering and no validation checkpoints, matching 'Steps missing or incoherent; no sequence; no validation for risky operations'.	1 / 5
Progressive Disclosure	The body is short and organized into clearly headed sections (Capabilities, Requirements, Patterns, Anti-Patterns, Sharp Edges, Related Skills) with no external references needed; minor gaps (empty anti-pattern headers, boilerplate 'When to Use') keep it just below 5, matching 'Good structure; most content is appropriately placed; minor organization gaps'.	4 / 5
	Total	9 / 20 Passed

Description

31%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is a second-person motivational preamble that names a domain but fails to state concrete capabilities or provide explicit usage triggers. It reads as persona framing rather than a functional skill description.

Suggestions

Rewrite in third person and lead with concrete actions the skill performs, e.g. 'Builds behavioral regression tests, capability assessments, and reliability metrics for LLM agents.'

Add an explicit 'Use when…' trigger clause, e.g. 'Use when evaluating LLM agents, diagnosing production failures, or designing agent test suites.'

Replace the narrative with natural trigger terms users actually say ('agent eval', 'flaky agent tests', 'benchmark vs production gap') to improve trigger-term coverage.

Dimension	Reasoning	Score
Specificity	The description only names the domain ('evaluating LLM agents') wrapped in motivational narrative ('You're a quality engineer who has seen agents that aced benchmarks fail spectacularly') with no concrete actions; the base anchor of 2 ('Names the domain but actions are minimal or generic') is reduced by 1 for pervasive second-person voice ('You're', 'You've learned').	1 / 5
Completeness	The 'what' is only vaguely implied through framing ('evaluating LLM agents is fundamentally different') and there is no 'Use when…' clause at all, fitting 'Has a vague what and no when'; the missing-trigger cap of 3 is not reached because the what itself is buried in narrative.	2 / 5
Trigger Term Quality	Relevant keywords appear ('LLM agents', 'benchmarks', 'production', 'testing') but common natural variations a user would say ('agent eval', 'flaky tests', 'regression suite', 'QA') are missing, matching 'Some relevant keywords but missing common variations or synonyms'.	3 / 5
Distinctiveness Conflict Risk	The LLM-agent-evaluation niche is somewhat specific, but the generic quality-engineer/benchmark framing ('agents that aced benchmarks fail spectacularly in production') could overlap with general testing or QA skills, matching 'Somewhat specific but could still overlap with similar skills'.	3 / 5
	Total	9 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: administrakt0r/AI-Agents-Safe-Coding-Skills
Path: plugins/AI-Agents-Safe-Coding-Skills/skills/agent-evaluation/SKILL.md
Commit: 900c504

Reviewed: 14 minutes ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.