CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

agent-evaluation

You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in production. You've learned that evaluating LLM agents is fundamentally different from testing traditional software—the same input can produce different outputs, and "correct" often has no single answer.

36

Quality

45%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./plugins/AI-Agents-Safe-Coding-Skills/skills/agent-evaluation/SKILL.md

The canonical home for this skill is agent-evaluation in sickn33/agentic-awesome-skills

SKILL.md
Quality
Evals
Security

Quality

Content

33%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is a hollow skeleton: it enumerates section headers and one-line hints but delivers no executable guidance, no sequenced workflow, and placeholder content (empty anti-patterns, comment-style 'solutions', a circular 'When to Use'). A reader would learn the topic's vocabulary but have no idea what to actually do.

Suggestions

Add a concrete evaluation workflow with sequenced steps and validation checkpoints, e.g. '1. Define behavioral invariants for the agent; 2. Run the suite N times (N≥10); 3. Analyze the pass-rate distribution, not a single run; 4. If variance is high, investigate flakiness before optimizing; 5. Re-run after any prompt/model change.'

Replace the comment-placeholder solutions in the Sharp Edges table with actual remedies (e.g. for flaky tests: 'run each test 10+ times and report pass-rate confidence intervals instead of pass/fail').

Flesh out or remove the empty Anti-Patterns section and the circular 'When to Use' filler, and delete the truncated persona intro — the body currently spends its budget on structure with almost no instruction.

DimensionReasoningScore

Conciseness

The body is short and mostly skeletal, but the ~80-word opening persona narrative duplicates the description and is truncated mid-sentence ('it's not 100% test pass rate—it'), which is unnecessary padding Claude does not need. This fits anchor 3 ('mostly efficient but includes some unnecessary explanation or could be tightened') rather than 4, because the narrative intro is a substantial waste, not a minor trim.

3 / 5

Actionability

Patterns give only high-level hints ('Run tests multiple times and analyze result distributions', 'Define and test agent behavioral invariants') with no concrete steps, code, or commands; the Sharp Edges 'solutions' are comment placeholders like '// Handle flaky tests in LLM agent evaluation'; and 'When to Use' is circular filler ('This skill is applicable to execute the workflow or actions described in the overview'). This matches anchor 2 (minimal concrete guidance, high-level hints, missing executable steps) rather than 1, because the Patterns and Sharp Edges table do name real, specific techniques and failure modes.

2 / 5

Workflow Clarity

No workflow exists at all: Patterns are unordered one-line descriptions, Anti-Patterns are bare headers with no content, and there are no steps, sequence, or validation checkpoints anywhere. This matches anchor 1 ('steps missing or incoherent; no sequence; no validation for risky operations') — an evaluation workflow (design suite → run N times → analyze distribution → check invariants → report) is entirely absent.

1 / 5

Progressive Disclosure

No bundle files exist and the body is under 50 lines with clear section headers (Capabilities, Requirements, Patterns, Anti-Patterns, Sharp Edges, Related Skills, When to Use) and no buried references, so the simple-skill exception applies. It scores 4 rather than 5 because several sections (Anti-Patterns, When to Use) are placeholder-empty, adding navigational noise rather than well-organized content.

4 / 5

Total

10

/

20

Passed

Description

31%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is persona-driven storytelling rather than a functional skill description: it establishes who the agent should be and a philosophy, but never states what the skill does or when to use it, uses second-person voice, and contains no concrete actions. Users searching for evaluation help would be poorly served by it as a trigger.

Suggestions

Rewrite in third person stating what the skill does with concrete actions, e.g. 'Designs and runs agent evaluations: behavioral regression suites, multi-run statistical testing, capability assessments, and reliability metrics. Use when the user mentions evals, agent testing, benchmark scores, flaky agent behavior, or agents failing in production.'

Add an explicit 'Use when...' trigger clause listing the natural phrases a user would say (e.g. 'evaluate my agent', 'agent evals', 'benchmark vs production gap', 'flaky tests').

Remove the persona narrative ('You're a quality engineer who has seen...') from the description — it consumes the description budget without adding actionable or trigger information.

DimensionReasoningScore

Specificity

The description lists no concrete actions — it is pure persona narrative ('You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in production') plus an abstract observation about non-determinism. It names the domain but contains zero actionable verbs, and the second-person voice ('You're...') triggers a 1-point specificity penalty per the judging guidelines, landing at 1 rather than 2.

1 / 5

Completeness

There is no 'Use when...' clause or any equivalent trigger guidance (which alone caps completeness at 3), and the 'what' is only vaguely implied through persona framing — the description never states what the skill actually does. This matches anchor 2 ('has a vague what and no when'), not 3, which requires a clearly stated 'what'.

2 / 5

Trigger Term Quality

Some relevant keywords appear ('agents', 'benchmarks', 'production', 'LLM agents', 'testing traditional software'), but common natural variations users would actually say ('evals', 'agent testing', 'regression testing', 'flaky tests') are missing, and the terms are buried in narrative rather than serving as triggers. This matches anchor 3, not 4, because keyword coverage is incomplete rather than merely missing a few terms.

3 / 5

Distinctiveness Conflict Risk

The domain (LLM agent evaluation) is somewhat specific, but the generic narrative framing ('quality engineer', 'benchmarks', 'production') could overlap with general testing, QA, and reliability skills. It fits anchor 3 — somewhat specific but still at overlap risk with similar skills — rather than 4, which requires mostly distinct triggers.

3 / 5

Total

9

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository

administrakt0r/AI-Agents-Safe-Coding-Skills

Path

plugins/AI-Agents-Safe-Coding-Skills/skills/agent-evaluation/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.