vector-index-tuning

Optimize vector index performance for latency, recall, and memory. Use when tuning HNSW parameters, selecting quantization strategies, or scaling vector search infrastructure.

1.56x

Quality

70%

Does it follow best practices?

Impact

100%

1.56x

Average score across 3 eval scenarios

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./tests/ext_conformance/artifacts/agents-wshobson/llm-application-dev/skills/vector-index-tuning/SKILL.md

The canonical home for this skill is vector-index-tuning in wshobson/agents

Quality

Content

52%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

Highly actionable code and concrete parameter guidance, but the skill is a monolithic wall of inlined templates with no progressive disclosure and no validation/feedback-loop checkpoints in its tuning workflow. Conciseness and structure are the main weaknesses.

Suggestions

Move the four large code templates into separate reference files (e.g., references/hnsw_tuning.py, references/quantization.py) and link to them from a concise overview, so SKILL.md stays lean.

Add an explicit tuning workflow with validation checkpoints (benchmark baseline -> change one parameter -> re-measure recall/latency -> only keep if it meets target), since benchmarking is iterative.

Trim explanatory boilerplate inside the templates (docstrings restating what quantization is) and keep only the executable specifics Claude cannot derive.

Dimension	Reasoning	Score
Conciseness	The body is ~520 lines dominated by four large full-class/function code templates that re-implement standard quantization and benchmarking logic Claude already knows; this is noticeably verbose and padded relative to a lean overview.	2 / 5
Actionability	Provides fully executable, copy-paste-ready code covering common cases (hnswlib benchmarking, INT8/PQ/binary quantization, Qdrant collection config, monitoring), with concrete parameter tables and memory estimation formulas.	5 / 5
Workflow Clarity	Presents an index-type selection guide and parameter tables giving rough sequencing, but there are no explicit validation checkpoints or feedback loops for the benchmarking/reindexing process despite 'Plan for maintenance' being a stated concern.	3 / 5
Progressive Disclosure	A single monolithic SKILL.md inlines hundreds of lines of API-style templates that clearly belong in separate reference files; there are no bundle files and no one-level-deep references to offload the bulk.	2 / 5
	Total	12 / 20 Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, third-person description that explicitly covers both what the skill does and when to use it, with domain-specific trigger phrases. Minor gaps in action breadth and synonyms keep specificity and trigger_term_quality at 4 rather than 5.

Dimension	Reasoning	Score
Specificity	Names the domain and lists several concrete actions ('tuning HNSW parameters', 'selecting quantization strategies', 'scaling vector search infrastructure'), though parameter tuning is the dominant action and coverage of other capabilities is lighter than a 5.	4 / 5
Completeness	Clearly answers 'what' (optimize vector index performance for latency, recall, and memory) and 'when' via an explicit 'Use when tuning HNSW parameters, selecting quantization strategies, or scaling vector search infrastructure.' clause with concrete trigger phrases.	5 / 5
Trigger Term Quality	Good natural terms a user would say ('HNSW parameters', 'quantization', 'vector search', 'latency', 'recall'), but missing common synonyms or file/library cues like 'ANN', 'similarity search', or 'Faiss/Qdrant'.	4 / 5
Distinctiveness Conflict Risk	Occupies a clear niche (vector index tuning) with distinct triggers (HNSW, quantization, vector search scaling) that are unlikely to collide with other skills.	5 / 5
	Total	18 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
skill_md_line_count	SKILL.md is long (524 lines); consider splitting into references/ and linking	Warning

	Total	15 / 16 Passed

Repository: Dicklesworthstone/pi_agent_rust
Path: tests/ext_conformance/artifacts/agents-wshobson/llm-application-dev/skills/vector-index-tuning/SKILL.md
Commit: 6453c0b

Reviewed: about 8 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.