hunt-rag-vector

Hunt vector-store / embedding-layer weaknesses in RAG pipelines (OWASP LLM08 Vector and Embedding Weaknesses) — persistent corpus poisoning that survives across sessions and users (distinct from one-shot indirect prompt injection, which is owned by hunt-llm-ai), cross-tenant vector-database IDOR (unauthenticated or unscoped queries against Pinecone/Weaviate/Chroma/Milvus/Qdrant/pgvector), source-text/metadata leakage in similarity-search results, and retrieval-hijack via adversarial embedding proximity ('SEO poisoning' for RAG). Targets: any app with a shared knowledge base, document upload feeding a chatbot, or a directly reachable vector-DB port. Validate: a second, clean session/account must inherit a poisoned result, or a cross-tenant artifact must be independently verifiable — confabulation is not a finding, same bar as hunt-llm-ai. Use when target is RAG-backed, exposes a vector-DB port, or lets users upload documents that other users' queries later retrieve.

Quality

92%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Critical

Do not install without reviewing

Quality

Content

85%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-organized, highly actionable hunting skill with executable probes, strong validation gates, and clear cross-skill chaining. Its only weakness is modest verbosity in framing prose that re-explains RAG/vector-DB fundamentals Claude already knows.

Suggestions

Tighten the Technique 2 and Technique 3 framing paragraphs to drop background about how RAG stores chunk text as metadata and where tenant isolation is usually enforced; keep only the operational 'probe this, prove that' guidance.

The Technique 1 'Embedding surface maximization' note could be condensed to a single imperative sentence since the retrieval concept is already understood.

Dimension	Reasoning	Score
Conciseness	Mostly efficient operational guidance, but several sentences explain concepts Claude already knows — e.g., 'RAG implementations almost universally store the original chunk text as metadata alongside the embedding vector' and the app-layer-vs-DB-layer tenant-isolation explanation — which could be tightened without losing the actionable signal.	2 / 3
Actionability	Provides concrete, executable curl and GraphQL probes, document-ID/namespace manipulation examples, and copy-pasteable proof-bar templates rather than abstract direction.	3 / 3
Workflow Clarity	Technique 1 is a numbered four-step sequence with explicit validation checkpoints (poll-until-ingested, second clean-session confirmation), reinforced by a dedicated False-Positive Gate and control-query rule for the destructive/cross-tenant operations.	3 / 3
Progressive Disclosure	A single well-sectioned file with clear headers and no nested (2+ level) references; sibling-skill mentions serve as navigation rather than buried detail, appropriate for a self-contained skill with no bundle files.	3 / 3
	Total	11 / 12 Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A dense, specific description that names concrete attack classes, named vector-DB products, and natural trigger phrases, while explicitly disambiguating from a sibling skill and providing a clear Use-when clause. It uses third-person voice and earns top marks on every dimension.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete attack actions — 'persistent corpus poisoning', 'cross-tenant vector-database IDOR (unauthenticated or unscoped queries against Pinecone/Weaviate/Chroma/Milvus/Qdrant/pgvector)', 'source-text/metadata leakage', and 'retrieval-hijack via adversarial embedding proximity' — matching the 'lists multiple specific concrete actions' anchor.	3 / 3
Completeness	Clearly answers both what (the four attack classes and validation bar) and when — 'Use when target is RAG-backed, exposes a vector-DB port, or lets users upload documents that other users' queries later retrieve.' — with an explicit Use-when clause.	3 / 3
Trigger Term Quality	Includes natural terms a security user would say — 'RAG pipelines', 'vector-DB port', 'document upload feeding a chatbot', 'shared knowledge base', and named vector stores — giving good coverage rather than just jargon.	3 / 3
Distinctiveness Conflict Risk	Explicitly carves a distinct niche from the sibling skill — 'distinct from one-shot indirect prompt injection, which is owned by hunt-llm-ai' — making wrong-skill triggering unlikely.	3 / 3
	Total	12 / 12 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: elementalsouls/Claude-BugHunter
Path: skills/hunt-rag-vector/SKILL.md
Commit: 6b9c96e

Reviewed: 2 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.