Content
62%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
The methodology core is excellent — a well-sequenced five-step process with explicit verification checkpoints and a revert-on-failure generalization loop — but it is undermined by redundant restatement sections, a complete absence of executable query examples in a skill about building CodeQL/Semgrep queries, and external references that point to files missing from the bundle.
Suggestions
Add the referenced files to the bundle (METHODOLOGY.md, resources/codeql/*.ql, resources/semgrep/*.yaml, resources/variant-report-template.md) or remove the references — currently every external pointer is a dead end.
Include at least one complete, executable CodeQL or Semgrep query example inline so the skill demonstrates real query syntax rather than only a placeholder rg command.
Cut the redundant "Key Principles" section and the "When to Use" list, which restate the description and Steps 1-5, to tighten token efficiency.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The step-by-step content is dense and non-pedagogical, but two sections are redundant: "When to Use" repeats the frontmatter description, and "Key Principles" re-states Step 1 ("Root cause first"), Step 2 ("Start specific"), Step 4's stop rule ("Know when to stop: 50%+ FP rate"), and Pitfall 1 ("Search everywhere"). This duplication is more than the minor over-explanation of the 4 anchor, fitting the "mostly efficient but could be tightened" 3 anchor. | 3 / 5 |
Actionability | The five-step process, abstraction decision table, and tool-selection matrix are concrete, but the only inline command is a placeholder (rg -n "exact_vulnerable_code_here") and no executable CodeQL or Semgrep query syntax appears anywhere — the "Ready-to-use templates" promised in resources/ are the missing key details for this query-building skill. More than minor gaps, so below the 4 anchor. | 3 / 5 |
Workflow Clarity | The five-step process is clearly sequenced with explicit validation ("Verify: Does it match exactly ONE location (the original)?"), a feedback loop with revert rule ("If FP rate too high, revert and try different abstraction"), an explicit stop condition ("Stop when false positive rate exceeds ~50%"), and a triage checklist (Location/Confidence/Exploitability/Priority) — matching the 5 anchor's explicit validation and error-recovery loops. The workflow is read-only search, so no destructive-operation cap applies. | 5 / 5 |
Progressive Disclosure | The body's structure is good — clear sections with one-level, clearly signaled references to METHODOLOGY.md and the resources/ tree — but scored against the actual bundle, every referenced path is dangling: no METHODOLOGY.md, no resources/ directory, and the only bundle file (assets/trail-of-bits-mark.svg) is never referenced. The navigation structure exists on paper but delivers nothing, fitting the 3 anchor's "references present but" non-functional organization. | 3 / 5 |
Total | 14 / 20 Passed |