hunt-rag-vector

Hunt vector-store / embedding-layer weaknesses in RAG pipelines (OWASP LLM08 Vector and Embedding Weaknesses) — persistent corpus poisoning that survives across sessions and users (distinct from one-shot indirect prompt injection, which is owned by hunt-llm-ai), cross-tenant vector-database IDOR (unauthenticated or unscoped queries against Pinecone/Weaviate/Chroma/Milvus/Qdrant/pgvector), source-text/metadata leakage in similarity-search results, and retrieval-hijack via adversarial embedding proximity ('SEO poisoning' for RAG). Targets: any app with a shared knowledge base, document upload feeding a chatbot, or a directly reachable vector-DB port. Validate: a second, clean session/account must inherit a poisoned result, or a cross-tenant artifact must be independently verifiable — confabulation is not a finding, same bar as hunt-llm-ai. Use when target is RAG-backed, exposes a vector-DB port, or lets users upload documents that other users' queries later retrieve.

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Critical

Do not install without reviewing

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured hunting skill that is dense, assumes competence, and pairs executable probing commands with strong validation gates and feedback loops for its batch/destructive operations. Its slight weaknesses are a few trimmable framing sentences and two techniques that read more as methodology than copy-paste guidance.

Dimension	Reasoning	Score
Conciseness	The body is dense and assumes Claude's competence — it never pads with what a vector DB, RAG, or embedding is — but a few framing sentences like 'Most RAG apps enforce tenant isolation in the application layer ... but not in the vector DB itself' could be trimmed, leaving it just shy of fully lean.	4 / 5
Actionability	It provides concrete, runnable commands (`curl -s http://$TARGET:8000/api/v1/heartbeat`, the Weaviate GraphQL POST, `GET /api/knowledge/document/00042`, `POST /api/chat` with attacker-supplied namespace) and a payload template, but Techniques 3 and 4 lean more on methodology/proof-bar language than copy-paste-ready commands.	4 / 5
Workflow Clarity	Technique 1 is a clearly numbered 4-step sequence with explicit validation and a feedback loop — 'Wait for ingestion', then from a 'second, unrelated session' confirm the OOB fires, with a failure branch ('If it only reproduces when you... that is not persistent poisoning') — and the False-Positive Gate is an explicit validation checklist for these destructive/batch operations.	5 / 5
Progressive Disclosure	No bundle files exist and all content is inline, but the document is well-organized under clear section headers with one-level-deep cross-skill references ('Read hunt-llm-ai's False-Positive Gate first', Related Skills & Chains); minor organization gaps (the severity table and related-skills list could live separately) keep it just below 5.	4 / 5
	Total	17 / 20 Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, highly specific description that names concrete attack classes and gives explicit 'Use when' triggers while disambiguating from the adjacent hunt-llm-ai skill. Its only weakness is a jargon-heavy register that slightly narrows the natural-phrase coverage.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete attack classes — 'persistent corpus poisoning that survives across sessions and users', 'cross-tenant vector-database IDOR (unauthenticated or unscoped queries against Pinecone/Weaviate/Chroma/Milvus/Qdrant/pgvector)', 'source-text/metadata leakage in similarity-search results', 'retrieval-hijack via adversarial embedding proximity' — giving comprehensive, specific coverage rather than generic language.	5 / 5
Completeness	It explicitly answers both 'what' (hunt vector-store/embedding-layer weaknesses) and 'when' ('Use when target is RAG-backed, exposes a vector-DB port, or lets users upload documents that other users' queries later retrieve'), with concrete trigger phrases for the 'when'.	5 / 5
Trigger Term Quality	The 'Use when target is RAG-backed, exposes a vector-DB port, or lets users upload documents that other users' queries later retrieve' clause plus 'shared knowledge base', 'document upload feeding a chatbot' are natural triggers, but the description leans heavily on technical jargon (OWASP LLM08, IDOR, adversarial embedding proximity) and omits a few common synonyms like 'semantic search' or 'chatbot' as standalone triggers, so it sits just below comprehensive.	4 / 5
Distinctiveness Conflict Risk	It carves out a clear niche and explicitly disambiguates from the closest neighbor — 'distinct from one-shot indirect prompt injection, which is owned by hunt-llm-ai' — so the triggers are unlikely to fire for the wrong skill.	5 / 5
	Total	19 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: elementalsouls/Claude-BugHunter
Path: skills/hunt-rag-vector/SKILL.md
Commit: d4b54e1

Reviewed: 7 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.