CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

testland/promptfoo-evaluation

Use when LLM prompts or RAG pipelines need eval-driven regression tests with Promptfoo: promptfooconfig.yaml providers, prompts and tests, deterministic, semantic, model-graded, latency and cost assertions, promptfoo eval and view, and CI gating.

Quality

88%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

82%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A highly actionable, well-structured reference: nearly all guidance is executable YAML or commands organized in a clear stepwise workflow with an anti-patterns table and honest limitations. Remaining gaps are a missing eval-failure feedback loop, an inline catalog that would fit better in a reference file, and a few version-sensitive details.

Suggestions

Add a short 'When evals fail' note (e.g., inspect promptfoo view output, loosen flaky deterministic asserts toward llm-rubric/similar, re-run) to close the feedback loop in the workflow.

Move provider API key setup (e.g., export OPENAI_API_KEY) from Step 8 into Step 2 so the first local promptfoo eval can actually run.

Split the Step 4 assertion catalog into a references/ file (e.g., references/assertions.md) and keep only the most common types inline to improve progressive disclosure and token efficiency.

DimensionReasoningScore

Conciseness

The body is dense and assumes competence (no explanations of YAML, CI, or embeddings), but the exhaustive deterministic-assertion table and version-pinned specifics like "Node.js 20.20+ or 22.22+" and "openai:gpt-5-mini-0613" are minor trims that keep it below the lean-everything-earns-its-place anchor.

4 / 5

Actionability

Everything is copy-paste ready: install commands, a complete minimal promptfooconfig.yaml, working interpolation/defaultTest/transform/assert-set YAML snippets, and the exact GitHub Action inputs including cache-path and no-share, covering the common cases the description promises.

5 / 5

Workflow Clarity

The eight steps (install → first eval → interpolation → assertions → defaultTest → transforms → assert-set → CI) are clearly sequenced with eval/view and CI gating as checkpoints, but there is no explicit failure-recovery loop and provider API keys are only introduced in Step 8 for CI, not before the first local eval.

4 / 5

Progressive Disclosure

Sections are well organized with a one-level-deep References list of real external doc URLs, but the ~60-line inline assertion catalog is content a bundle reference file would ideally hold, and the link-reference definitions sit at the top of the file before any prose, leaving minor organization gaps.

4 / 5

Total

17

/

20

Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it states what the skill does in concrete, jargon-appropriate terms and pairs it with an explicit "Use when" trigger clause. The only weaknesses are a few missing natural synonyms and slight overlap risk with neighboring eval-framework skills.

DimensionReasoningScore

Specificity

Enumerates multiple concrete capabilities — "promptfooconfig.yaml providers, prompts and tests", "deterministic, semantic, model-graded, latency and cost assertions", "promptfoo eval and view", "CI gating" — comprehensively covering the skill's surface, matching the anchor for multiple specific concrete actions rather than several actions with minor gaps.

5 / 5

Completeness

Explicitly answers both questions: an explicit "Use when LLM prompts or RAG pipelines need eval-driven regression tests" trigger clause plus a concrete enumeration of what the skill does, matching the anchor where both what and when are clearly and explicitly stated with concrete trigger phrases.

5 / 5

Trigger Term Quality

Includes natural phrases users would say — "LLM prompts", "RAG pipelines", "regression tests", "promptfooconfig.yaml", "CI gating" — but misses common synonyms such as "prompt testing", "LLM evals", or "golden tests", so it fits good-but-not-comprehensive keyword coverage.

4 / 5

Distinctiveness Conflict Risk

The tool name "Promptfoo" and the "promptfooconfig.yaml" filename give it a clear niche trigger, but the general "RAG pipelines need eval" phrasing overlaps with closely related sister skills the body itself names (ragas-evaluation, openai-evals), so minor overlap risk remains.

4 / 5

Total

18

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

On this page