CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

testland/ragas-evaluation

Use for any question or task about RAG or agent evaluation metrics, including which metrics to use, scoring an answer against its contexts, and telling retrieval problems from generation problems after a change such as chunk size: Ragas faithfulness, response relevancy, context precision and recall, noise sensitivity, factual correctness, tool-call accuracy, agent goal accuracy and rubric scoring.

Quality

88%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

78%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured skill body: a clear step sequence, executable code for install, custom metrics, dataset construction, and CI, with the metric catalog properly offloaded to a real references file. The main improvement opportunities are trimming quickstart boilerplate and wiring the judge-LLM configuration into the primary evaluate() example.

Suggestions

Trim the Step 2 verbatim quickstart to just the DiscreteMetric construction and ascore call, dropping the asyncio main/__main__ scaffolding that adds tokens without teaching the pattern.

Show the judge-LLM setup (llm_factory) in the Step 6 evaluate() example so the primary evaluation path is fully copy-paste runnable end-to-end, not just the custom-metric path.

Fold the anti-patterns' error-recovery guidance (e.g., NaN from missing retrieved_contexts) into the Step 4 dataset-shape step as an explicit 'if a metric returns NaN, check the required field' checkpoint.

DimensionReasoningScore

Conciseness

The body is mostly efficient: the 'The model:' summary, the dataset-field table, and the anti-patterns table all earn their tokens, and there is no re-explanation of concepts Claude already knows. It sits at anchor 4 rather than 5 because the verbatim Step 2 quickstart carries boilerplate (the asyncio main/__main__ scaffolding) that could be trimmed to just the DiscreteMetric pattern.

4 / 5

Actionability

Concrete, executable guidance throughout: install commands, a complete DiscreteMetric example, an EvaluationDataset.from_list example, and a CI snippet with threshold asserts. It matches anchor 4 rather than 5 because the judge-LLM setup (llm_factory) is shown only for the custom-metric path and never wired into the primary evaluate() flow, leaving a minor gap in copy-paste readiness for the main use case.

4 / 5

Workflow Clarity

A clear six-step sequence (Install, metric pattern, pick metrics, dataset shape, framework integration, CI) with threshold asserts as checkpoints and an anti-patterns table that covers error recovery (e.g., NaN when retrieved_contexts is missing). This is anchor 4 rather than 5 because validation guidance is preventive and tabular rather than an explicit inline validate-then-fix-then-retry loop within the step sequence.

4 / 5

Progressive Disclosure

The body is an overview that moves the 30+ metric catalog into references/metrics.md (verified to exist), clearly signaled with a markdown link and source attribution, exactly one level deep. This matches the top anchor for a clear overview with well-signaled, one-level-deep references and easy navigation.

5 / 5

Total

17

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: concrete capabilities, an explicit 'Use for...' trigger clause with natural phrasing, and a comprehensive metric enumeration that clearly distinguishes it from sister evaluation skills. The only gap is a few missing natural synonyms such as 'hallucination' or 'groundedness'.

DimensionReasoningScore

Specificity

The description lists multiple concrete actions ('scoring an answer against its contexts', 'telling retrieval problems from generation problems after a change such as chunk size', 'which metrics to use') and comprehensively enumerates the metric names (faithfulness, response relevancy, context precision and recall, noise sensitivity, factual correctness, tool-call accuracy, agent goal accuracy). This matches the anchor for multiple specific concrete actions with comprehensive coverage; it is not the level below because coverage of the domain's capabilities is thorough rather than minor-gapped.

5 / 5

Completeness

It explicitly answers 'when' with a 'Use for...' clause containing concrete trigger phrases ('any question or task about RAG or agent evaluation metrics', 'after a change such as chunk size') and 'what' via the enumerated scoring capabilities and metric names. This matches the top anchor for both what and when with concrete triggers; it is not the level below because the 'when' is explicit and specific rather than merely adequate.

5 / 5

Trigger Term Quality

Good natural keyword coverage: 'RAG', 'agent evaluation metrics', 'faithfulness', 'context precision and recall', 'chunk size', and the tool name 'Ragas' itself. It falls between the 4 and 5 anchors because a few natural terms users would plausibly say are missing (e.g., 'hallucination', 'groundedness', 'RAG eval'), so it lacks the synonym-level comprehensiveness of the top anchor.

4 / 5

Distinctiveness Conflict Risk

It carves a clear niche (RAG/agent evaluation via Ragas) with triggers anchored on Ragas-specific metric names (noise sensitivity, agent goal accuracy, tool-call accuracy), and the skill body explicitly scopes sister skills to non-RAG prompt evals and pytest-native evals. Conflict risk is minimal, matching the top anchor; it is not the level below because the enumerated metric names make mistaken triggering unlikely.

5 / 5

Total

19

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

On this page