Content
38%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
The body delivers a genuinely executable Wanda quick start and useful comparison tables, but it is padded with basic pruning-concept explanations, its later strategies and production pipeline rely on undefined pseudocode helpers, and it inlines a large Wanda deep-dive that duplicates an existing references/wanda.md that is never linked. Main fixes: link and defer to the reference file, make the strategy/pipeline code executable, and add a validate-before-deploy checkpoint.
Suggestions
Reference references/wanda.md explicitly (e.g., a '## Wanda deep dive — See [references/wanda.md](references/wanda.md)' section) and move the duplicated Wanda criterion/algorithm details out of SKILL.md, cutting the body substantially.
Make the strategy and production-pipeline code executable: define or remove undefined helpers (train_step, fine_tune, prune_layer, prune_model, load_calibration_data, finetune_dataset), import torch.nn.functional as F for nm_prune, and pass calibration data in the format wanda_prune expects.
Add a validation checkpoint to the workflow: evaluate the pruned model (e.g., lm-eval on arc_easy/hellaswag) against a degradation threshold before saving/deploying, with a retry (lower sparsity or fine-tune) if the threshold is exceeded.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The 485-line body includes several padded sections explaining concepts Claude already knows: 'Structured vs Unstructured' pros/cons lists, 'Sparsity Patterns' bit-mask illustrations ('[1, 0, 1, 0, 1, 1, 0, 0]... Pros: Flexible, high quality'), a baseline 'Magnitude Pruning' snippet, and pseudo-code if-statements in 'Method Selection' ('if no_retraining_budget: use_method = "wanda"'). The Wanda criterion is also explained twice (full quick-start code plus a 'Core Concepts' re-explanation). This matches anchor 2 ('several unnecessary explanations or padded sections') rather than 3, where only a small amount of trimming would be needed. | 2 / 5 |
Actionability | The Wanda quick start is concrete and near-executable ('def wanda_prune(model, calib_data, sparsity=0.5)...'), but many later blocks are pseudocode with undefined helpers: 'train_step(model)', 'prune_layer(module, sparsity)', 'fine_tune(model, epochs=2, lr=1e-5)', 'prune_model(model, sparsity=...)', 'load_calibration_data()', and an undefined 'finetune_dataset' in the production pipeline. 'nm_prune' uses 'F.pad' without importing torch.nn.functional, the pipeline passes a HF Dataset where wanda_prune expects a list of strings, and the evaluation snippet subtracts result dicts ('(original_results - pruned_results)'). This is the anchor-3 mix of concrete guidance with pseudocode and missing key details, not 4, whose code is mostly executable with only minor gaps. | 3 / 5 |
Workflow Clarity | The production pipeline lists a numbered sequence ('1. Load model... 2. Load calibration data... 3. Apply pruning... 4. (Optional) Fine-tune... 5. Save') and an Evaluation section exists, but pruning is a destructive, in-place batch modification of all weights ('W *= mask.float()') with no integrated validation checkpoint — the evaluation of pruned vs original is a separate section, not a step with a threshold or error-recovery loop. Per the rubric's cap for destructive/batch operations without validation, this cannot exceed 3. | 3 / 5 |
Progressive Disclosure | The bundle contains references/wanda.md (347 lines covering Wanda's algorithm, calibration, deployment, and results) that is never mentioned in SKILL.md — it is undiscoverable — while the body inlines substantial duplicated Wanda material (full quick-start code, 'Core Concepts' pruning criteria, resources) that clearly belongs in that reference file. Scoring against the actual bundle structure, this matches anchor 2 ('content that clearly belongs in separate files is inlined; or references are buried'): the reference exists but is never signaled, and ~500 lines of content that should be split are inline. Not 1 because the body itself has clear section headers; not 3 because there is no clearly signaled reference at all and the body is monolithic. | 2 / 5 |
Total | 10 / 20 Passed |