CtrlK
BlogDocsLog inGet started
Tessl Logo

aatmf-t12-rag-poisoning

AATMF T12 — RAG & Knowledge Base Manipulation. PoisonedRAG, vector store flood, embedding collision, retrieval-bias attacks.

49

Quality

54%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Critical

Do not install without reviewing

Fix and improve this skill with Tessl

tessl review fix ./packages/decepticon/decepticon/skills/plugins/llm-redteam/t12-rag-poisoning/SKILL.md
SKILL.md
Quality
Evals
Security

Quality

Content

58%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is concise, well-structured, and appropriately scoped for a compact catalog skill, scoring well on conciseness and progressive disclosure. It is held to 3 on actionability and workflow clarity because techniques are described conceptually without executable code or explicit validation checkpoints for the destructive/batch operations involved.

Suggestions

Add at least one runnable code snippet for a representative technique (e.g. a small script computing embedding similarity or submitting a canary document) to lift actionability above conceptual description.

Insert explicit validation/checkpoint steps into the probe workflow (e.g. 'Verify canary was ingested before querying', 'Confirm retrieval occurred before checking execution').

Make the end-to-end flow explicit: probe → observe detection signal → classify severity → recommend defender control, so the multi-step process is sequenced rather than presented as parallel sections.

DimensionReasoningScore

Conciseness

The body is tight and assumes Claude's competence — it does not explain what RAG or embeddings are, and each technique is a compact bullet block. A few telegraphic fragments ('Bypass-by-displacement', 'Lower bandwidth but harder to detect') are slightly informal but not padded; minor tightening possible but largely efficient.

4 / 5

Actionability

Guidance is concrete in places (numbered probe steps, a YAML probe pattern, a severity table, defender controls) but the core techniques are described as attack concepts rather than executable steps — there is no runnable code beyond the YAML config snippet, and 'Engineer embedding via gradient descent against the embedding model' is a high-level hint, not an executable recipe.

3 / 5

Workflow Clarity

The custom probe section gives a clear 3-step sequence (upload canary doc → issue query → check execution), which is good, but it lacks explicit validation/verification checkpoints and the broader skill has no end-to-end workflow tying probe → detection signal → severity together; per the rubric, destructive/batch-style operations (poison submission, flood) without validation cap this at 3.

3 / 5

Progressive Disclosure

No bundle files exist, but the body is under 50 lines and well-organized into clearly signaled sections (Techniques, Probe pattern, Detection signals, Severity, Defender, Cross-references), so it does not need external references; the single academic citation is appropriate and one-level. Structure is good with only minor gaps (the probe YAML and the technique list could arguably live in reference files if the skill grew).

4 / 5

Total

14

/

20

Passed

Description

50%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description effectively names a specific security niche and enumerates concrete attack vectors, giving it reasonable specificity and distinctiveness. It is capped at 3 on completeness because it lacks any explicit 'Use when...' trigger guidance, and trigger-term quality is jargon-heavy rather than user-natural.

Suggestions

Add an explicit 'Use when...' clause, e.g. 'Use when assessing RAG or knowledge-base security, testing retrieval-layer manipulation, or reviewing vector-store isolation.'

Reframe the description in third-person action voice stating what the skill does (e.g. 'Tests and documents RAG manipulation techniques...') rather than only listing attack names.

Include a couple of natural user-facing synonyms (e.g. 'RAG security', 'retrieval poisoning') alongside the technical terms.

DimensionReasoningScore

Specificity

Names the domain (RAG/knowledge-base manipulation) and lists concrete attack types ('PoisonedRAG, vector store flood, embedding collision, retrieval-bias attacks'), but these are categories rather than concrete actions the skill performs — it does not state what Claude does (detect? remediate? simulate?) beyond naming attack vectors.

3 / 5

Completeness

It gives a clear 'what' (lists the attack classes) but has no 'Use when...' clause or explicit trigger guidance — per the rubric, a missing explicit trigger guidance caps completeness at 3.

3 / 5

Trigger Term Quality

Contains relevant terms ('RAG poisoning', 'vector store', 'embedding collision', 'retrieval bias', 'semantic search') but these are technical jargon leaning toward attacker-side framing; the natural phrasings a defender/user would say (e.g. 'RAG security', 'retrieve misuse') are partially covered but no synonyms or file extensions are present.

3 / 5

Distinctiveness Conflict Risk

The T12/RAG-manipulation niche is fairly specific and unlikely to conflict with unrelated skills; minor overlap risk exists with adjacent prompt-injection or training-poisoning skills (cross-referenced as T1/T6), but the RAG-vector-store framing is distinct enough.

4 / 5

Total

13

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

metadata_version

'metadata.version' is missing

Warning

Total

15

/

16

Passed

Repository
PurpleAILAB/Decepticon
Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.