aatmf-t12-rag-poisoning

AATMF T12 — RAG & Knowledge Base Manipulation. PoisonedRAG, vector store flood, embedding collision, retrieval-bias attacks.

Quality

66%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Critical

Do not install without reviewing

Fix and improve this skill with Tessl

tessl review fix ./packages/decepticon/decepticon/skills/plugins/llm-redteam/t12-rag-poisoning/SKILL.md

Quality

Content

65%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is concise and well-organized as a tactical attack catalog, but it stays mostly descriptive with a procedural probe flow and no executable scripts or external reference files, which limits actionability, workflow feedback, and progressive disclosure.

Suggestions

Add executable probe scripts (e.g., a script that uploads a canary doc, issues a query, and asserts canary execution) to move actionability toward copy-paste-ready.

Add an explicit validate→fix→retry feedback loop in the probe workflow so failed canary detection drives a corrective next step.

Move the per-technique details (T12.001–T12.007) into a one-level-deep reference file, keeping SKILL.md as an overview, to improve progressive disclosure.

Dimension	Reasoning	Score
Conciseness	The body is lean and terse — short bullets, a one-line RAG definition, and minimal explanation of concepts Claude already knows — so it matches the 'lean and efficient; assumes Claude's competence' anchor rather than the padded/mostly-efficient levels below.	3 / 3
Actionability	It provides some concrete guidance (a copy-paste YAML probe block and a 3-step poison-doc submission flow), but the submission flow is procedural rather than executable code/commands and lacks runnable scripts, fitting 'some concrete guidance but incomplete; pseudocode instead of executable code'.	2 / 3
Workflow Clarity	The custom probe lays out a clear upload→query→check sequence, but the 'check if canary executed' step has no fix/retry feedback loop or explicit validation checkpoint, matching 'steps listed but validation gaps; checkpoints missing or implicit'.	2 / 3
Progressive Disclosure	The skill is a single well-sectioned file with no nested references, but seven detailed technique writeups plus severity/detection tables are kept inline with no bundle files or one-level-deep reference split, fitting 'some structure but content that should be separate is inline'.	2 / 3
	Total	9 / 12 Passed

Description

67%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific and well-scoped to a distinct niche, but it is jargon-heavy and lacks any explicit 'Use when...' trigger guidance, capping completeness and trigger-term quality.

Suggestions

Add an explicit trigger clause such as 'Use when testing RAG/knowledge-base security, retrieval poisoning, or vector-store abuse' to raise completeness.

Soften jargon (e.g., 'embedding collision') with natural terms users would actually say ('semantic search manipulation', 'poisoned retrieved documents') to improve trigger-term quality.

Dimension	Reasoning	Score
Specificity	The description names the domain ('RAG & Knowledge Base Manipulation') and lists four concrete named techniques ('PoisonedRAG, vector store flood, embedding collision, retrieval-bias attacks'), matching the 'lists multiple specific concrete actions' anchor; it is not merely 'names domain and some actions' (level 2).	3 / 3
Completeness	It clearly answers 'what' (T12 RAG manipulation techniques) but provides no 'Use when...' clause or equivalent explicit trigger guidance, which the guidelines say should cap completeness at 2 even though the 'what' is solid.	2 / 3
Trigger Term Quality	Terms like 'RAG', 'vector store', and 'retrieval-bias' are relevant but skew technical; 'embedding collision' is pure jargon and common natural variations a user would actually say are missing, so it fits 'some relevant keywords but missing common variations' rather than full natural-term coverage (3) or jargon-only (1).	2 / 3
Distinctiveness Conflict Risk	The 'AATMF T12' coding plus the narrow RAG/knowledge-base niche make it clearly distinguishable and unlikely to trigger for unrelated skills, matching the 'clear niche with distinct triggers' anchor; the slight adjacency to T1 prompt injection does not push it down to level 2.	3 / 3
	Total	10 / 12 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
metadata_version	'metadata.version' is missing	Warning

	Total	15 / 16 Passed

Repository: PurpleAILAB/Decepticon
Path: packages/decepticon/decepticon/skills/plugins/llm-redteam/t12-rag-poisoning/SKILL.md
Commit: 4484f85

Reviewed: about 15 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.