CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/promptfoo-evaluation

Authors and runs Promptfoo evals for LLM prompts and RAG pipelines - wires `promptfooconfig.yaml` providers + prompts + tests + assertions (deterministic `equals` / `contains` / `is-json` / `regex`, semantic `similar`, model-graded `llm-rubric` / `factuality` / `g-eval`, performance `latency` / `cost`, custom `javascript` / `python`), runs `npx promptfoo eval`, views HTML report via `promptfoo view`, and integrates CI for regression gating. Use when the user runs Promptfoo, asks about prompt regression suites, or needs an eval-driven workflow for LLM-backed features.

68

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

78%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A highly actionable, well-sequenced skill whose commands and config examples are copy-paste ready, with a useful anti-patterns table and honest limitations section. The main gap is structural: the assertion catalog and metric thresholds belong in a references/ file so SKILL.md can stay a lean overview, and the workflow lacks an explicit verify-before-gate step.

Suggestions

Move the full assertion catalog (deterministic, text-quality, model-graded tables and default thresholds) into a references/assertions.md file and keep only 3-4 representative examples inline, so SKILL.md reads as an overview.

Add an explicit validation step to the workflow, e.g., 'After promptfoo eval, check the exit code / open promptfoo view and confirm all asserts pass before committing the config or gating CI.'

Trim the repeated inline citation parentheticals ('Per [pf-gh][pf-gh]', '(Commands per [pf-gh][pf-gh])') and consolidate link definitions at the bottom to reduce token overhead.

DimensionReasoningScore

Conciseness

The body is dense with commands, YAML snippets, and tables, and explains nothing Claude already knows. Minor over-explanation remains: repeated 'Per [pf-gh][pf-gh]' / '(Commands per [pf-gh][pf-gh])' citation parentheticals and the inline default-thresholds table could be trimmed. Fits 'Efficient; minor instances of over-explanation' rather than the every-token-earns-its-place anchor at 5.

4 / 5

Actionability

Everything is executable: install commands, 'promptfoo init --example getting-started' + eval + view sequence, a complete minimal promptfooconfig.yaml, working assertion YAML for each category, and a full GitHub Actions 'with:' block. Copy-paste ready across all common cases, matching the top anchor.

5 / 5

Workflow Clarity

Steps 1-8 form a clear install-to-CI sequence, but there is no explicit verification checkpoint (e.g., inspect the HTML report or eval exit code before treating the gate as passed). Fits 'Clear sequence with most checkpoints present; minor validation gaps' rather than 5, which requires explicit validation steps and feedback loops; the destructive/batch cap of 3 does not apply.

4 / 5

Progressive Disclosure

No bundle files exist, and the full assertion catalog, text-quality metric thresholds, and CI action details are inlined in a ~280-line SKILL.md rather than split into references/ files. External doc links are well signaled in a References section, but reference material that clearly belongs in separate files is inline, matching the 'some structure; could be better organized' anchor rather than 4.

3 / 5

Total

16

/

20

Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: concrete, third-person, tool-specific, with an explicit 'Use when' trigger clause and an unusually detailed enumeration of assertion types. Its only weaknesses are slightly thin natural-language keyword variations and a generic final trigger clause that could collide with sister eval skills.

DimensionReasoningScore

Specificity

The description lists multiple concrete actions ('Authors and runs Promptfoo evals', 'wires promptfooconfig.yaml providers + prompts + tests + assertions', 'runs npx promptfoo eval', 'views HTML report via promptfoo view', 'integrates CI for regression gating') with an enumerated assertion-type catalog, matching the comprehensive-coverage anchor. It is not 4 because no meaningful capability of the skill is left unnamed.

5 / 5

Completeness

Clearly answers 'what' (authors/runs evals, wires config components, runs eval, views report, integrates CI) and an explicit 'Use when...' clause with concrete trigger phrases. This matches the anchor requiring both what and when with concrete triggers; it is not 4 because the 'when' is already explicit and specific.

5 / 5

Trigger Term Quality

Good natural keywords ('runs Promptfoo', 'prompt regression suites', 'RAG pipelines', 'LLM-backed features') but common variations users would say, such as 'evaluate prompts', 'prompt testing', or 'LLM evals', are missing. Fits the 'good keyword coverage; a few natural terms missing' anchor rather than the comprehensive synonym coverage of 5.

4 / 5

Distinctiveness Conflict Risk

The tool name 'Promptfoo' and file trigger 'promptfooconfig.yaml' make it mostly distinct, but the generic clause 'needs an eval-driven workflow for LLM-backed features' overlaps with closely related eval skills (openai-evals, deepeval, ragas are named as sister tools in the body). Fits 'mostly distinct; minor overlap risk' better than the minimal-conflict anchor at 5.

4 / 5

Total

18

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

Table of Contents