agent-evaluation
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.
68
Does it follow best practices?
If you maintain this skill, you can automatically optimize it using the tessl CLI to improve its score:
npx tessl skill review --optimize ./path/to/skillValidation for skill structure
Discovery
100%Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.
This is a strong skill description that follows best practices. It clearly specifies the domain (LLM agent testing), lists concrete capabilities, includes an explicit 'Use when:' clause with natural trigger terms, and carves out a distinct niche. The description is appropriately concise while being comprehensive.
| Dimension | Reasoning | Score |
|---|---|---|
Specificity | Lists multiple specific concrete actions: 'behavioral testing, capability assessment, reliability metrics, and production monitoring'. Also includes a concrete detail about benchmark performance ('less than 50% on real-world benchmarks'). | 3 / 3 |
Completeness | Clearly answers both what (testing and benchmarking LLM agents with specific methods) AND when (explicit 'Use when:' clause with trigger terms). The structure follows the recommended pattern. | 3 / 3 |
Trigger Term Quality | Includes natural keywords users would say: 'agent testing', 'agent evaluation', 'benchmark agents', 'agent reliability', 'test agent'. These cover common variations of how users would phrase requests about LLM agent testing. | 3 / 3 |
Distinctiveness Conflict Risk | Clear niche focused specifically on 'LLM agents' testing rather than general testing or general LLM work. The combination of 'agent' + 'testing/benchmarking' creates distinct triggers unlikely to conflict with code testing or general LLM skills. | 3 / 3 |
Total | 12 / 12 Passed |
Implementation
22%Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
This skill is essentially a skeleton or outline rather than actionable guidance. It names important concepts in agent evaluation (statistical testing, behavioral contracts, adversarial testing) but provides no concrete implementation details, code examples, or step-by-step workflows. The Sharp Edges table has placeholder comments instead of actual solutions, and the description is truncated mid-sentence.
Suggestions
Add concrete, executable code examples for each pattern (e.g., a Python snippet showing how to run statistical test evaluation with multiple runs and confidence intervals)
Complete the Sharp Edges solutions with actual actionable guidance instead of placeholder comments
Provide a clear step-by-step workflow for setting up and running an agent evaluation suite, including validation checkpoints
Finish the truncated description and remove the narrative framing ('You're a quality engineer...') in favor of direct instructions
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The content is relatively brief but includes some unnecessary framing ('You're a quality engineer who has seen agents...') and the description is cut off mid-sentence. The patterns and anti-patterns sections are too sparse to be useful. | 2 / 3 |
Actionability | The skill provides no concrete code, commands, or executable examples. Patterns like 'Statistical Test Evaluation' and 'Behavioral Contract Testing' are named but not explained with any implementation details. The Sharp Edges table has empty solutions (just comments). | 1 / 3 |
Workflow Clarity | There is no clear workflow or sequence of steps for evaluating agents. The content lists concepts and categories but provides no guidance on how to actually perform agent evaluation, run tests, or interpret results. | 1 / 3 |
Progressive Disclosure | The content has some structure with clear sections (Patterns, Anti-Patterns, Sharp Edges, Related Skills), but the sections are incomplete stubs. References to related skills exist but no actual linked documentation or deeper resources are provided. | 2 / 3 |
Total | 6 / 12 Passed |
Validation
90%Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.
Validation — 10 / 11 Passed
Validation for skill structure
| Criteria | Description | Result |
|---|---|---|
frontmatter_unknown_keys | Unknown frontmatter key(s) found; consider removing or moving to metadata | Warning |
Total | 10 / 11 Passed | |
- Reviewed
Table of Contents
Is this your skill?
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.