promptfoo-evaluation
Configures and runs LLM evaluation using Promptfoo framework. Use when setting up prompt testing, creating evaluation configs (promptfooconfig.yaml), writing Python custom assertions, implementing llm-rubric for LLM-as-judge, or managing few-shot examples in prompts. Triggers on keywords like "promptfoo", "eval", "LLM evaluation", "prompt testing", or "model comparison".
Install with Tessl CLI
npx tessl i github:fernandezbaptiste/claude-code-skills --skill promptfoo-evaluation89
Does it follow best practices?
If you maintain this skill, you can automatically optimize it using the tessl CLI to improve its score:
npx tessl skill review --optimize ./path/to/skillValidation for skill structure
Discovery
100%Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.
This is an excellent skill description that hits all the marks. It provides specific concrete actions, includes explicit trigger keywords, clearly answers both what and when, and uses distinctive terminology that minimizes conflict risk with other skills. The description uses proper third-person voice throughout.
| Dimension | Reasoning | Score |
|---|---|---|
Specificity | Lists multiple specific concrete actions: 'setting up prompt testing', 'creating evaluation configs (promptfooconfig.yaml)', 'writing Python custom assertions', 'implementing llm-rubric for LLM-as-judge', 'managing few-shot examples in prompts'. | 3 / 3 |
Completeness | Clearly answers both what ('Configures and runs LLM evaluation using Promptfoo framework') and when ('Use when setting up prompt testing...') with explicit trigger guidance including a 'Triggers on keywords' clause. | 3 / 3 |
Trigger Term Quality | Explicitly lists natural trigger keywords users would say: 'promptfoo', 'eval', 'LLM evaluation', 'prompt testing', 'model comparison'. Also includes technical terms like 'promptfooconfig.yaml' and 'llm-rubric' that users familiar with the tool would use. | 3 / 3 |
Distinctiveness Conflict Risk | Highly distinctive with specific tool name 'Promptfoo', specific file format 'promptfooconfig.yaml', and domain-specific terms like 'llm-rubric' and 'LLM-as-judge' that clearly distinguish it from generic testing or evaluation skills. | 3 / 3 |
Total | 12 / 12 Passed |
Implementation
72%Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
This is a strong, actionable skill with excellent executable examples covering the full Promptfoo workflow. The main weaknesses are some unnecessary explanatory content (the overview paragraph, the real-world example with local paths) and missing validation checkpoints in workflows - particularly important given that LLM evaluations can be expensive API operations.
Suggestions
Remove the Overview paragraph - Claude knows what Promptfoo is; start directly with Quick Start
Add a validation step before running eval: suggest using echo provider first to verify config, or add 'npx promptfoo@latest validate' if available
Remove or generalize the 'Real-World Example' section - the local path (/Users/tiansheng/...) is not useful and the structure is already shown elsewhere
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The skill is mostly efficient with good code examples, but includes some unnecessary sections like the 'Overview' paragraph explaining what Promptfoo is (Claude knows this), and the 'Real-World Example' section with a specific local path that adds little value. | 2 / 3 |
Actionability | Excellent executable code throughout - complete YAML configs, Python assertion functions with proper return types, bash commands, and JSON prompt formats. All examples are copy-paste ready with realistic patterns. | 3 / 3 |
Workflow Clarity | The Quick Start provides a clear sequence, but the skill lacks explicit validation checkpoints. For example, there's no guidance on verifying config syntax before running expensive API calls, or validating Python assertions work before full evaluation runs. | 2 / 3 |
Progressive Disclosure | Well-structured with clear sections progressing from Quick Start to Core Configuration to Advanced patterns. References external file (references/promptfoo_api.md) appropriately for detailed API docs, keeping the main skill focused. | 3 / 3 |
Total | 10 / 12 Passed |
Validation
100%Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.
Validation — 11 / 11 Passed
Validation for skill structure
No warnings or errors.
- Reviewed
Table of Contents
Is this your skill?
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.