Content
71%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
The skill body is a well-sequenced, largely executable penetration-testing playbook with real verification steps (MIC checks, FCnt detection, evidence hashing) and appropriate OPSEC cautions for destructive operations. Its main weaknesses are the monolithic single-file layout with no reference files, a pseudocode-only final phase, and a trimmable background intro.
Suggestions
Move the frequency table, the AES session-key derivation and FRMPayload encryption recipes, and the ChirpStack rogue-gateway setup into references/ files (e.g., references/crypto-recipes.md, references/chirpstack-setup.md), keeping SKILL.md as a phase-by-phase overview with one key command per phase.
Make Phase 6 executable: provide a runnable script that takes the captured ciphertext, byte offset, and flip mask, recalculates the MIC with NwkSKey, and outputs the modified frame instead of placeholder pseudocode.
Add explicit validation checkpoints before destructive phases (e.g., confirm MIC verification passes and FCnt-check status is confirmed before running the replay, and verify the target DevEUI/AppKey pair resolves in ChirpStack before enqueueing the downlink).
Trim the introductory blockquote and paper citations down to one or two lines, since Claude already knows LoRaWAN's deployment context.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The body is dense and command/code-forward with comments that earn their place, but contains trimmable background explanation Claude does not need: the opening blockquote ('LoRaWAN is deployed globally for smart meters, asset tracking, agriculture, and industrial sensors...') and asides like 'This is documented in Toothpick / LoRaWAN security analysis papers.' This matches the anchor 'Efficient; minor instances of over-explanation that could be trimmed' — the padding is real but localized, so it is not the level-3 'mostly efficient but includes some unnecessary explanation' profile. | 4 / 5 |
Actionability | Most guidance is copy-paste executable: 'rtl_sdr -f 868100000 -s 1000000 -g 40', the gr-lora decode command, LAF scan/replay invocations, and complete Python for MIC verification, session-key derivation, and FRMPayload encryption. Gaps are minor: Phase 6 is placeholder pseudocode ('ciphertext[TARGET_BYTE] ^= FLIP_MASK' with undefined variables), and Phase 5 steps 2-3 ('Register a gateway...', 'Register the target device...') are comment-only directions. This fits 'Mostly executable guidance; concrete code or commands with minor gaps' rather than the fully-executable level 5. | 4 / 5 |
Workflow Clarity | The six phases give a clear capture -> parse -> derive -> replay -> inject -> modify sequence, with verification steps present (MIC verification function in Phase 3, the FCnt-check detection procedure in Phase 4, and sha256 evidence hashing), plus OPSEC gating ('Gate replay tests on explicit operator authorization'). It falls short of level 5 because there are no explicit validate-before-proceeding checkpoints between the destructive phases and Phase 5's setup steps are vague, matching 'Clear sequence with most checkpoints present; minor validation gaps'. The destructive-operation cap at 3 does not apply since verification steps do exist. | 4 / 5 |
Progressive Disclosure | The body is well-sectioned with clear headers, but it is a single ~250-line monolithic file with no bundle files: content that clearly belongs in separate references (the regional frequency table, the full AES session-key/encryption recipes, the ChirpStack setup steps) is inlined, and the References section points to external URLs rather than navigating any internal material. This matches the anchor 'Some structure but could be better organized; content that should be separate is inline' — it is not level 2 (structure is genuinely good) but not level 4 (nothing is split out or signaled as separable). | 3 / 5 |
Total | 15 / 20 Passed |