vector-index-tuning
Optimize vector index performance for latency, recall, and memory. Use when tuning HNSW parameters, selecting quantization strategies, or scaling vector search infrastructure.
80
71%
Does it follow best practices?
Impact
100%
1.56xAverage score across 3 eval scenarios
Passed
No known issues
Optimize this skill with Tessl
npx tessl skill review --optimize ./tests/ext_conformance/artifacts/agents-wshobson/llm-application-dev/skills/vector-index-tuning/SKILL.mdQuality
Discovery
100%Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.
This is an excellent skill description that concisely covers specific capabilities, uses appropriate domain-specific trigger terms, and clearly separates the 'what' from the 'when'. It uses proper third-person voice and targets a well-defined niche that minimizes conflict risk with other skills.
| Dimension | Reasoning | Score |
|---|---|---|
Specificity | Lists multiple specific concrete actions: 'optimize vector index performance', 'tuning HNSW parameters', 'selecting quantization strategies', 'scaling vector search infrastructure'. These are concrete, actionable capabilities. | 3 / 3 |
Completeness | Clearly answers both what ('Optimize vector index performance for latency, recall, and memory') and when ('Use when tuning HNSW parameters, selecting quantization strategies, or scaling vector search infrastructure') with explicit trigger guidance. | 3 / 3 |
Trigger Term Quality | Includes strong natural keywords a user would use: 'vector index', 'latency', 'recall', 'memory', 'HNSW parameters', 'quantization', 'vector search'. These are the exact terms someone working in this domain would mention. | 3 / 3 |
Distinctiveness Conflict Risk | Highly distinctive niche — vector index optimization with specific terms like HNSW, quantization strategies, and vector search infrastructure are unlikely to conflict with other skills. This is a clearly defined technical domain. | 3 / 3 |
Total | 12 / 12 Passed |
Implementation
42%Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
The skill provides highly actionable, executable code templates for vector index tuning but suffers from extreme verbosity—most of the code is standard implementation that Claude could generate without instruction. The content lacks a coherent workflow connecting the templates and has no progressive disclosure structure, dumping everything into one massive file. The most valuable parts are the concise reference tables (index selection, HNSW parameters, quantization types) which are unfortunately overshadowed by boilerplate code.
Suggestions
Extract the four code templates into separate referenced files (e.g., templates/hnsw_tuning.py, templates/quantization.py) and keep only the decision tables, parameter recommendations, and brief usage notes in SKILL.md.
Add an explicit tuning workflow: benchmark current performance → identify bottleneck (latency/recall/memory) → select strategy → apply changes → validate improvement → iterate if needed.
Remove code that Claude can generate from context (e.g., the full VectorQuantizer class, VectorSearchMonitor class) and replace with concise parameter guidance and decision criteria.
Add validation checkpoints to the workflow, such as 'verify recall@10 ≥ target before deploying' and 'compare latency p99 against SLA after parameter changes'.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The skill is extremely verbose at ~400+ lines, with extensive code templates that Claude could generate on its own. The quantization class, monitoring class, and Qdrant configuration are all standard patterns that don't need to be spelled out in full. The reference tables (index selection, HNSW parameters, quantization types) are concise and valuable, but they're buried in massive code blocks. | 1 / 3 |
Actionability | The code templates are fully executable with real libraries (hnswlib, qdrant_client, sklearn), include proper imports, type hints, and concrete parameter values. The benchmark function, quantization strategies, and Qdrant configuration are all copy-paste ready. | 3 / 3 |
Workflow Clarity | While individual templates are clear, there's no overall workflow connecting them (e.g., 'first benchmark → then select parameters → then quantize → then monitor'). There are no validation checkpoints or feedback loops for the tuning process—no guidance on what to do when recall drops below target or when latency exceeds thresholds. | 2 / 3 |
Progressive Disclosure | All content is inlined in a single monolithic file with no references to supporting files. The four large code templates should be in separate files, with SKILL.md providing a concise overview, the decision tables, and links to template files. The external resource links at the bottom are helpful but don't compensate for the lack of internal structure. | 1 / 3 |
Total | 7 / 12 Passed |
Validation
90%Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.
Validation — 10 / 11 Passed
Validation for skill structure
| Criteria | Description | Result |
|---|---|---|
skill_md_line_count | SKILL.md is long (524 lines); consider splitting into references/ and linking | Warning |
Total | 10 / 11 Passed | |
- Repository
- Dicklesworthstone/pi_agent_rust
- Commit
bbc5ade
- Reviewed
Table of Contents
Is this your skill?
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.