CtrlK
BlogDocsLog inGet started
Tessl Logo

promptfoo-evaluation

Configures and runs LLM evaluation using Promptfoo framework. Use when setting up prompt testing, creating evaluation configs (promptfooconfig.yaml), writing Python custom assertions, implementing llm-rubric for LLM-as-judge, or managing few-shot examples in prompts. Triggers on keywords like "promptfoo", "eval", "LLM evaluation", "prompt testing", or "model comparison".

88

1.59x
Quality

82%

Does it follow best practices?

Impact

97%

1.59x

Average score across 3 eval scenarios

SecuritybySnyk

Advisory

Suggest reviewing before use

SKILL.md
Quality
Evals
Security

Quality

Content

64%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

This is a solid, highly actionable Promptfoo skill with excellent concrete examples covering configuration, assertions, prompt formats, and troubleshooting. Its main weaknesses are length — several advanced sections (long text handling, real-world example, advanced few-shot) could be offloaded to referenced files — and the lack of an integrated validation workflow tying the pieces together. The troubleshooting section with common gotchas (maxConcurrency placement, file:// resolution, relay 401 errors) is particularly valuable.

Suggestions

Move 'Long Text Handling', 'Real-World Example', and 'Advanced Few-Shot Implementation' sections into separate referenced files to reduce the main SKILL.md to a focused overview (~150 lines).

Add an explicit validation workflow: e.g., '1. Write config → 2. Run with echo provider to verify prompts → 3. Run single test case → 4. Run full eval → 5. Review results with `promptfoo view`' with checkpoints at each step.

DimensionReasoningScore

Conciseness

The skill is fairly comprehensive but includes some unnecessary verbosity. Sections like 'Long Text Handling' with a Chinese content curation example and the 'Real-World Example' section add bulk that could be in referenced files. Some explanations (e.g., 'open-source CLI tool for testing and comparing LLM outputs') are unnecessary for Claude. However, most content is practical and not padded with basic concept explanations.

2 / 3

Actionability

Excellent actionability throughout — nearly every section contains executable code, complete YAML configs, working Python functions with proper signatures and return types, and specific CLI commands. The assertion table, file reference patterns, and troubleshooting section all provide concrete, copy-paste-ready guidance.

3 / 3

Workflow Clarity

The Quick Start provides a clear 3-step sequence, and the troubleshooting section addresses common failure modes with solutions. However, there's no explicit validation workflow for the overall eval setup process — no 'verify config before running' step, no feedback loop for when evals fail or produce unexpected results. The echo provider section partially addresses this but isn't integrated into a cohesive workflow.

2 / 3

Progressive Disclosure

The skill references `references/promptfoo_api.md` and a `./tiaogaoren/` example project, showing awareness of progressive disclosure. However, the bundle has no files, so these references are unverifiable. More importantly, the SKILL.md itself is quite long (~300+ lines) with sections like the full real-world example, long text handling, and advanced few-shot patterns that could be split into separate referenced files rather than inlined.

2 / 3

Total

9

/

12

Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

This is an excellent skill description that hits all the marks. It provides specific concrete actions, comprehensive trigger terms, explicit 'Use when' and 'Triggers on' clauses, and is highly distinctive due to its focus on the Promptfoo framework. The description uses proper third-person voice and is concise without being vague.

DimensionReasoningScore

Specificity

Lists multiple specific concrete actions: configuring and running LLM evaluation, setting up prompt testing, creating evaluation configs (promptfooconfig.yaml), writing Python custom assertions, implementing llm-rubric for LLM-as-judge, and managing few-shot examples in prompts.

3 / 3

Completeness

Clearly answers both 'what' (configures and runs LLM evaluation using Promptfoo framework, creates configs, writes assertions, etc.) and 'when' (explicit 'Use when...' clause and 'Triggers on keywords like...' clause with specific trigger terms).

3 / 3

Trigger Term Quality

Excellent coverage of natural trigger terms including 'promptfoo', 'eval', 'LLM evaluation', 'prompt testing', 'model comparison', 'promptfooconfig.yaml', 'llm-rubric', 'custom assertions', and 'few-shot examples'. These are terms users would naturally use when needing this skill.

3 / 3

Distinctiveness Conflict Risk

Highly distinctive with a clear niche around the Promptfoo framework specifically. The mention of specific artifacts like 'promptfooconfig.yaml', 'llm-rubric', and the Promptfoo tool name make it very unlikely to conflict with other skills.

3 / 3

Total

12

/

12

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation11 / 11 Passed

Validation for skill structure

No warnings or errors.

Repository
daymade/claude-code-skills
Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.