Content
50%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
The body is genuinely actionable — exact commands, concrete sink/helper examples, a precise output contract, and well-branched workflow paths — but it is severely overlong. The discovery checklist compresses a handful of general principles into ~60 dense, family-specific bullets with heavy internal repetition that would be far more token-efficient split into reference files, and the workflow lacks an explicit post-emission validation checkpoint.
Suggestions
Collapse the ~60-bullet Discovery Checklist into a short set of general principles (enumerate concrete instances; keep seeded/advisory rows open until closed on the same proof tuple; never trade a row for a louder neighboring issue; one root cause = one candidate) and move the family-specific rules (deserialization codecs, SSO/SAML validators, archive extraction, realm/authenticator classes, query APIs, operation/action runners) into per-family reference files linked one level deep.
Rewrite the remaining checklist bullets as terse imperatives; e.g., "independently reachable sink/control = separate candidate" instead of 60-word multi-clause sentences that restate the same rule in different vocabulary.
Add an explicit validation checkpoint after candidate emission: re-read the candidate ledger and confirm every candidate has a stable id, a discovery receipt, and a complete proof tuple before finishing, so the workflow has a verify-and-fix loop rather than an implicit one.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The Discovery Checklist runs ~60 dense, multi-clause bullets that restate a handful of core principles many times over: instance-splitting appears as "keep each vulnerable instance addressable" (line 51), "its own candidate instance" (57), "needs its own candidate id" (63), and "separate candidate instances" (62), while "do not drop a row in favor of a louder neighboring issue" recurs at lines 82, 89, and 90. It never explains concepts Claude already knows, which keeps it above anchor 1, but the padding and redundancy are extensive. | 2 / 5 |
Actionability | The body supplies exact commands with flags ("generate_rank_input.py make-diff-rank-input --repo <repo_root> --base <base> --mode revisions --head <head> --out <discovery_dir>/rank_input.jsonl"), a concrete output contract enumerating required fields per candidate, and concrete search hints ("Version", "versionCompare", "getDOM", "cloneNode", "execute/executemany/executescript"). It misses anchor 5 because commands contain unresolved placeholders (<python_command>, <plugin_dir>) and a large share of the checklist is abstract directive rather than executable guidance. | 4 / 5 |
Workflow Clarity | The entry conditions are clearly branched (compact inventory, code diff, standalone repository discovery) and the Code Diff Workflow lists ordered steps, with one explicit checkpoint ("If a required input is still missing, stop and ask the user for it before continuing") and a termination criterion ("Continue reviewing until no additional distinct plausible candidates remain"). However, the dominant operational guidance is a flat 60-item rule list with no feedback loop validating emitted candidates or the ledger, which fits anchor 3 (sequence present but checkpoints missing or implicit) rather than 4. | 3 / 5 |
Progressive Disclosure | External references are clearly signaled with purpose and are one level deep ("read `../../references/artifact-storage.md` and follow its storage policy", "defer to `../attack-path-analysis/references/severity-policy.md`"), and the body has clear section headers. But no bundle files ship with this skill, and the family-specific enumeration rules (SAML/SSO validators, XML parser factories, archive extraction, realm/authenticator packages, query APIs) are ~60 bullets of inlined material that belongs in per-family reference files — the anchor-3 case of content that should be separate staying inline. | 3 / 5 |
Total | 12 / 20 Passed |