CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/deepeval-evaluation

Authors and runs DeepEval - pytest-native LLM eval framework with `LLMTestCase` (input + actual_output + expected_output + retrieval_context) and ~11 built-in metrics (G-Eval, Answer-Relevancy, Faithfulness, Contextual-Recall / Precision / Relevancy, Hallucination, Bias, Toxicity, Summarization, JSON-Correctness); runs via `deepeval test run {file.py}` with `assert_test()` per test or `evaluate()` for batch; integrates Confident-AI dashboard. Use when the user prefers pytest workflow, works with RAG and needs faithfulness/contextual metrics out-of-the-box, or wants a managed dashboard.

70

Quality

88%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

82%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is a dense, executable guide: verbatim quickstart, install/run/CI commands, metric and field tables, and a strong anti-patterns section, with almost no filler. Remaining gaps are the missing validate-fix-retry loop around batch evaluation and a small amount of citation/quotation scaffolding that could be trimmed or moved to a reference file.

Suggestions

Add a short feedback loop after Step 6/7: when a case scores below threshold, read the metric's reasoning, fix the case or prompt, and re-run — this supplies the missing validation checkpoint for batch operations.

Trim the repeated 'Per [de-gh][de-gh]' citation asides to a single Sources section, and make the batch example concrete by constructing the cases (or loading them) instead of referencing undefined `case1` variables.

DimensionReasoningScore

Conciseness

The body is lean — tables for fields and metrics, two tight code blocks, an anti-patterns table — and never explains concepts Claude already knows (pytest, RAG). The repeated 'Per [de-gh][de-gh]' / 'Per [de-start][de-start]' citation scaffolding and the quoted marketing sentence ('an open-source LLM eval package...') are minor instances of over-explanation that could be trimmed, placing it at the level-4 anchor rather than the fully lean level-5.

4 / 5

Actionability

Install command, verbatim executable quickstart, run command, a second executable GEval snippet, and the CI invocation (`deepeval test run tests/llm/ --run-async --workers 4`) are all copy-paste ready and cover the common cases (first test, custom metric, batch, CI). The only illustration-not-runnable line is `evaluate(test_cases=[case1, case2, case3], ...)` with undefined variables, which is too minor to drop it from the level-5 anchor to level 4.

5 / 5

Workflow Clarity

Steps 1-7 are clearly sequenced (install → first test → fields → metrics → custom metric → batch → CI) and include some checkpoints (threshold passes if score ≥ threshold; 'pytest exit code propagates: nonzero if any assert_test fails'), and the anti-patterns table supplies error-avoidance guidance. There is no explicit validate→fix→retry loop for the batch evaluation path, which is the minor validation gap that distinguishes the level-4 anchor from level 5.

4 / 5

Progressive Disclosure

The single-file body is well sectioned with clear headers, link-reference definitions at the top, and a References section; navigation is easy. The metric catalog and LLMTestCase field tables are compact and reasonably inline, though a skill of ~175 lines could split the metric catalog into a reference file — a minor organization gap matching the level-4 anchor rather than the well-signaled multi-file level-5 pattern.

4 / 5

Total

17

/

20

Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is dense, specific, and written in third person with a complete 'what' plus an explicit three-condition 'Use when' clause. Its only weaknesses are a few missing natural trigger synonyms (e.g., 'evaluate my LLM app', 'LLM testing') and modest overlap with other RAG-evaluation skills on the RAG/faithfulness trigger.

Suggestions

Add one or two everyday phrasings users would actually type (e.g., 'evaluate my LLM app' or 'LLM testing/regression testing') alongside the pytest/RAG triggers.

Sharpen the RAG trigger so it fires specifically when a pytest workflow is preferred (e.g., 'works with RAG AND wants metrics inside pytest') to reduce overlap with Ragas-style skills.

DimensionReasoningScore

Specificity

"Authors and runs DeepEval", "runs via `deepeval test run {file.py}` with `assert_test()` ... or `evaluate()` for batch" names multiple concrete actions plus the concrete artifacts (LLMTestCase fields, ~11 named metrics, Confident-AI dashboard). Coverage of the framework's capability surface is comprehensive, matching the level-5 anchor rather than the 'several specific actions, minor gaps' level-4 anchor.

5 / 5

Completeness

The 'what' is explicit (authors/runs DeepEval tests with listed metrics, commands, and APIs) and the 'when' is an explicit "Use when the user prefers pytest workflow, works with RAG ..., or wants a managed dashboard" clause with three concrete trigger conditions. This matches the level-5 anchor (both what and when clearly answered with concrete trigger phrases); the level-4 anchor's 'when could be more explicit or specific' does not apply.

5 / 5

Trigger Term Quality

Natural trigger phrases like "prefers pytest workflow", "works with RAG", "needs faithfulness/contextual metrics", "managed dashboard" are present and would be said by real users. Common variations such as "evaluate my LLM app", "LLM testing", or "regression testing" are missing, so it falls between the level-4 (good coverage, a few natural terms missing) and level-5 (comprehensive including synonyms) anchors, and the level-4 anchor is the closer fit.

4 / 5

Distinctiveness Conflict Risk

"pytest-native", "assert_test()", "Confident-AI dashboard" carve a clear niche distinct from generic eval skills, and the triggers differentiate it from sister skills. However 'works with RAG and needs faithfulness/contextual metrics' overlaps with a Ragas-style RAG-eval skill, so there is minor overlap risk with closely related skills — the level-4 anchor rather than the minimal-conflict level-5 anchor.

4 / 5

Total

18

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

Table of Contents