CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

rag-eval

Filesystem RAG benchmarks: corpus/, train.json, evaluate_rag.py (RAGAS quality). Not for prod monitoring, latency/throughput benchmarking (use rag-perf), or evals outside this repo layout.

Quality

79%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./skills/rag-eval/SKILL.md
SKILL.md
Quality
Evals
Security

Quality

Content

78%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable overview with excellent progressive disclosure into a real, one-level-deep reference bundle. The main costs are redundancy — the Agent playbook duplicates the Instructions section and several gotchas are stated twice — and the lack of a proactive pre-flight validation step in the workflow.

Suggestions

Merge the 'Agent playbook' section into 'Instructions' (or replace Instructions with it): both enumerate the same five steps (run, tune, convert, analyze, triage), cutting roughly 8 lines of duplication in conciseness.

State the rag-perf deflection once (e.g., in 'When not to use') and drop the repetition in 'Purpose' and 'Limitations'; similarly state the `/v1` ingestor-URL rule once instead of in both the Troubleshooting table and Gotchas.

Add an explicit pre-flight validation checkpoint before running the eval — e.g., 'Verify RAG/ingestor endpoints respond and ingestion completed (row count matches train.json) before scoring' — to lift workflow clarity with a proactive rather than purely reactive feedback loop.

DimensionReasoningScore

Conciseness

The body is dense and free of concept explanations Claude already knows, but it carries noticeable duplication: the 'Agent playbook' section re-states the five 'Instructions' steps (run eval, tune quality, conversion, analyze, triage), the rag-perf deflection appears in both 'Purpose' and 'When not to use', and the '/v1' ingestor-URL rule appears in both the Troubleshooting table and Gotchas. That places it at anchor 3 ('mostly efficient but could be tightened') rather than 4, since consolidating whole redundant sections is more than a minor trim.

3 / 5

Actionability

Fully executable, copy-paste-ready commands: `uv sync --project scripts/eval`, `uv run --project scripts/eval python scripts/eval/evaluate_rag.py --dataset-paths ... --host localhost --port 8081`, and `python3 -m json.tool results/<dataset>/rag_<dataset>_evaluation_summary.json`, plus concrete error-to-fix mappings ('Pass `http://host:port` only — code appends `/v1/`', 'Use `--collection` with a unique name'). Placeholders like `/path/to/my_dataset` are appropriately justified, matching anchor 5.

5 / 5

Workflow Clarity

A clear five-step sequence (prepare data → run eval → tune quality → analyze results → triage errors) with an error-signal table and recovery guidance (stale collections, empty contexts, fewer rows than train.json). It sits at anchor 4 rather than 5 because validation checkpoints are mostly reactive/triage-style rather than explicit pre-flight gates — e.g., there is no 'verify servers are reachable / ingestion completed before running the eval' step — and ingestion/eval runs are batch-style operations where a proactive checkpoint would be warranted.

4 / 5

Progressive Disclosure

The body is a genuine overview: procedural bulk (CLI flags, conversion patterns, full error tables, analysis scripts) is pushed to four real reference files (all verified to exist), each linked with purpose-scoped anchors, and none of them nest further references — exactly one level deep. The 'Source of truth' table additionally acts as a navigation map, matching anchor 5's 'clear overview with well-signaled one-level-deep references'.

5 / 5

Total

17

/

20

Passed

Description

78%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, dense description that names concrete artifacts, the quality metric, and explicit exclusions with cross-skill routing. Its only gap is the absence of a positive 'Use when...' trigger clause, which keeps completeness and trigger quality at 4 rather than 5.

DimensionReasoningScore

Specificity

The description lists several concrete, specific items — 'Filesystem RAG benchmarks: corpus/, train.json, evaluate_rag.py (RAGAS quality)' — naming exact files and the quality metric. It stops short of anchor 5's comprehensiveness because the actions themselves (prepare, run, tune, triage) are only implied by 'benchmarks', and it is above anchor 3 because multiple concrete artifacts and the metric are explicitly named, not just the domain.

4 / 5

Completeness

The 'what' is clear and concrete (filesystem RAG benchmarks with named artifacts and RAGAS quality), and the 'when' is addressed through equivalent explicit scoping: 'Not for prod monitoring, latency/throughput benchmarking (use rag-perf), or evals outside this repo layout'. This fits anchor 4 ('both what and when; when could be more explicit') rather than 5, because there is no positive 'Use when...' trigger clause, and not 3, because the negative boundary guidance is explicit and specific rather than weakly implied.

4 / 5

Trigger Term Quality

Good natural keyword coverage: 'RAG benchmarks', 'corpus/', 'train.json', 'evaluate_rag.py', 'RAGAS quality', 'latency/throughput benchmarking', 'prod monitoring' — terms a user working in this repo would plausibly say. It falls short of anchor 5 (comprehensive synonyms/extensions) because common phrasings like 'evaluate quality', 'retrieval quality', or 'NVIDIA RAG' are absent, but it is well above anchor 3's 'some relevant keywords'.

4 / 5

Distinctiveness Conflict Risk

The description carves out a clear niche (filesystem RAG benchmarks fixed to this repo layout) and explicitly routes adjacent use cases elsewhere — '(use rag-perf)' for latency/throughput, exclusions for prod monitoring and non-repo evals. This matches anchor 5's 'clear niche with distinct triggers; minimal conflict risk'; voice is third-person noun-phrase style with no person penalty.

5 / 5

Total

17

/

20

Passed

Validation

68%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 11 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

allowed_tools_field

'allowed-tools' contains unusual tool name(s)

Warning

metadata_version

'metadata.version' is missing

Warning

metadata_field

'metadata' should map string keys to string values

Warning

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

referenced_paths_exist

Referenced path issues: 15 missing, 8 deeper-than-1-level

Warning

Total

11

/

16

Passed

Repository

NVIDIA/skills

Path

skills/rag-eval/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.