Name: testland/deepeval-evaluation
Rating: 73.6 (1 reviews)
Author: testland

testland/deepeval-evaluation

Authors and runs DeepEval - pytest-native LLM eval framework with `LLMTestCase` (input + actual_output + expected_output + retrieval_context) and ~11 built-in metrics (G-Eval, Answer-Relevancy, Faithfulness, Contextual-Recall / Precision / Relevancy, Hallucination, Bias, Toxicity, Summarization, JSON-Correctness); runs via `deepeval test run {file.py}` with `assert_test()` per test or `evaluate()` for batch; integrates Confident-AI dashboard. Use when the user prefers pytest workflow, works with RAG and needs faithfulness/contextual metrics out-of-the-box, or wants a managed dashboard.

Quality

92%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

80%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An actionable, lean guide with executable code and well-organized sections, but it keeps all reference material (metric catalog, field tables) inline in a single ~180-line file and omits explicit validation/feedback checkpoints for its batch and CI operations.

Suggestions

Add an explicit validation checkpoint to the batch/CI workflow — e.g., a 'if assert_test fails: inspect the metric's reasoning, adjust criteria or test case, re-run' feedback loop — so batch operations have a clear verify->fix->retry sequence.

Move the 11-metric catalog and the LLMTestCase field table into a separate reference file (e.g. references/metrics.md) referenced one level deep from SKILL.md, keeping the body as a concise overview that earns the progressive_disclosure top score.

Surface a one-line pass/fail verification cue inside Step 2 and Step 7 (not just the anti-patterns table) so the validation logic sits in the workflow sequence itself.

Dimension	Reasoning	Score
Conciseness	Lean step-by-step body with code and tables that assumes Claude's competence — it does not explain what pytest, metrics, or LLMs are; not a 2 because there is no padded unnecessary explanation to trim.	3 / 3
Actionability	Fully executable, copy-paste-ready guidance throughout — verbatim quickstart test, install/run commands, GEval custom pattern, and batch evaluate() code; not a 2 because the examples are complete rather than pseudocode or missing key details.	3 / 3
Workflow Clarity	Steps 1-7 are clearly sequenced, but batch evaluation and CI runs (batch operations) lack an explicit validate->fix->retry checkpoint loop; verification is only implicit via thresholds and exit-code propagation, so it caps at 2 rather than 3.	2 / 3
Progressive Disclosure	Well-organized into clear sections, but monolithic: the 11-metric catalog and LLMTestCase field tables are inline reference material that could live in separate bundle files, and only external URL references exist with no internal bundle structure; not a 3 because content that should be split is inline, not a 1 because sections are well organized and references are one level deep.	2 / 3
	Total	10 / 12 Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, third-person description that states concrete capabilities with named APIs and pairs them with an explicit multi-condition 'Use when' trigger clause covering pytest, RAG faithfulness metrics, and managed dashboard needs.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions with specific APIs — 'Authors and runs', 'runs via deepeval test run', 'assert_test() per test or evaluate() for batch', 'integrates Confident-AI dashboard' — matching the 'lists multiple specific concrete actions' anchor; not below because it is comprehensive rather than partial.	3 / 3
Completeness	Explicitly answers both what ('Authors and runs DeepEval - pytest-native LLM eval framework...') and when ('Use when the user prefers pytest workflow, works with RAG...') with explicit triggers, in third person; not below because neither half is missing or merely implied.	3 / 3
Trigger Term Quality	Natural terms a user would actually say are well covered — 'pytest workflow', 'RAG', 'faithfulness/contextual metrics', 'managed dashboard'; not a 2 because common variations are present rather than missing.	3 / 3
Distinctiveness Conflict Risk	Clear niche (pytest-native DeepEval with Confident-AI dashboard) and distinct triggers make it unlikely to fire for the wrong skill; not below because it is not merely 'somewhat specific' — the triggers are sharply scoped.	3 / 3
	Total	12 / 12 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

7 days ago

Table of Contents

Discovery Implementation Validation