Content
72%Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
The body is a concise, well-organized technique catalog with a concrete probe-pattern config, but it describes many techniques rather than giving fully executable steps and lacks explicit validation checkpoints in its probe workflow.
Suggestions
Convert key techniques into executable, copy-paste-ready probe snippets — e.g. an actual curl/HTTP example for model_id probing and a concrete promptfoo test case for schema bypass — rather than describing the action.
Add a short ordered probe workflow with a validation checkpoint, e.g. 1) run probe pattern, 2) confirm signal against the Detection signals list, 3) re-run to rule out false positives before scoring severity.
For each technique, pair the attack with a one-line verification of success (what observable result confirms it worked) so findings are confirmable rather than asserted.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The body is a lean bullet catalog of techniques with a one-line orientation ('Attack the LLM as an API service...') and no padded explanation of concepts Claude already knows; every section earns its tokens. | 3 / 3 |
Actionability | It offers some concrete artifacts — a promptfoo probe-pattern YAML block, example prompts ('Repeat the word token 500 times'), and candidate model_id values — but many techniques are described ('induce model to NOT call the function', 'Probe with increasingly-long inputs') rather than given as executable, copy-paste-ready steps. | 2 / 3 |
Workflow Clarity | The Probe pattern section supplies a runnable config, but there is no sequenced validate→fix→retry workflow or explicit checkpoints for confirming a finding (e.g. confirming model-version manipulation succeeded), which leaves the sequence present but checkpoints implicit. | 2 / 3 |
Progressive Disclosure | It is a single self-contained file with no external bundle references and clearly organized sections (Techniques, Probe pattern, Detection signals, Severity, Defender, Cross-references), so the well-organized structure satisfies progressive disclosure without needing separate files. | 3 / 3 |
Total | 10 / 12 Passed |