CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/solr-relevance-tests

Tests Apache Solr search relevance by querying a test core, asserting ranking and score expectations, uploading LTR feature stores and models via the `/schema/feature-store` and `/schema/model-store` REST APIs, using `debugQuery` for per-document score explain, tuning eDisMax parameters (`qf`, `pf`, `mm`, `bq`), and computing judgment-driven nDCG checks against pinned corpora. Use when the search stack runs Apache Solr (enterprise, SolrCloud, or embedded) and you need a pre-deploy relevance gate or LTR model verification.

76

Quality

96%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

92%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable and well-sequenced with strong validation feedback loops, and it is concise in the sense of never restating basics. Its main weakness is progressive disclosure: it is a single monolithic document rather than an overview pointing to one-level-deep reference files.

Suggestions

Extract the larger code/config blocks (e.g. full LTR features.json, the CI workflow, the eDisMax parameter reference) into files under references/ and link to them one level deep, keeping SKILL.md as a tighter overview.

Move the per-step detail for LTR (Steps 8-9) and eDisMax tuning (Step 7) into a references/solr-ltr.md and references/edismax-tuning.md, leaving a short summary plus a link in the main body.

Add a short 'Quick start' or 'When to use' pointer at the top that links to the detail files so the overview reads as a navigation hub rather than a wall of steps.

DimensionReasoningScore

Conciseness

The body is dense and code-forward — every step leads with executable Python/bash/YAML and never explains fundamentals Claude already knows (what Solr or nDCG is); the intro only differentiates from ES/OS, which is load-bearing, not padding.

3 / 3

Actionability

All guidance is copy-paste-ready executable code: real query params, curl uploads to /schema/feature-store and /schema/model-store, working nDCG/assert functions, and a complete CI workflow, matching the score-3 anchor.

3 / 3

Workflow Clarity

A clear 10-step sequence with explicit validation checkpoints (assert mean >= 0.75, per-query -10% guards, feature-store-before-model ordering) and an Anti-patterns table reinforcing feedback loops, matching the score-3 anchor.

3 / 3

Progressive Disclosure

The skill is a monolithic 10-step guide with everything inline and no bundle files (references/scripts/assets absent); detailed material like the full LTR features.json and CI workflow lives in SKILL.md rather than being split into one-level-deep reference files, matching the score-2 anchor of 'content that should be separate is inline'.

2 / 3

Total

11

/

12

Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific, complete, and distinctive: it enumerates concrete Solr-specific capabilities, gives an explicit 'Use when...' trigger, and is clearly separated from the Elasticsearch/OpenSearch relevance skills. It is dense with domain vocabulary but every term is genuine trigger language for this need.

DimensionReasoningScore

Specificity

The description lists multiple concrete actions — querying a test core, asserting ranking/score expectations, uploading LTR feature stores and models via named REST APIs, using debugQuery, tuning eDisMax params, and computing nDCG — matching the score-3 anchor for listing several specific concrete actions.

3 / 3

Completeness

It explicitly answers both what ('Tests Apache Solr search relevance by...') and when via an explicit 'Use when the search stack runs Apache Solr... and you need a pre-deploy relevance gate or LTR model verification' clause, matching the score-3 anchor.

3 / 3

Trigger Term Quality

It includes natural trigger vocabulary a user with this need would say ('Apache Solr', 'search relevance', 'pre-deploy relevance gate', 'LTR model verification') with good coverage, matching the score-3 anchor; the technical tokens (qf, pf, nDCG) are the genuine domain vocabulary rather than generic jargon.

3 / 3

Distinctiveness Conflict Risk

It carves a clear Solr-specific niche (debugQuery explain, LTR feature/model store REST, eDisMax tuning) distinct from the ES/OS _rank_eval approach, making it unlikely to trigger for the wrong skill.

3 / 3

Total

12

/

12

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

metadata_version

'metadata.version' is missing

Warning

Total

15

/

16

Passed

Reviewed

Table of Contents

DiscoveryImplementationValidation