Content
82%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
A strong, highly actionable skill: every phase has executable commands, validation gates (user confirmation, count verification), and genuine non-obvious domain knowledge that justifies its presence. Main improvement areas are tightening the verbose provider-quirk prose, adding an explicit recovery branch to Phase 5, and offloading reference-grade detail to a separate file.
Suggestions
Add an explicit feedback loop to Phase 5: state what to do when the deprecated=True count does not match expectations (e.g., re-run the grep per model_id and investigate discrepancies before proceeding).
Move the detailed provider-quirk explanations (Cloudflare aliasing, LiteLLM/models.dev warnings, Fireworks detail-API logic, OpenRouter suffix handling) into a references/ file, keeping only a one-line summary per provider in SKILL.md.
Tighten the Cloudflare paragraph into discrete bullets (auth fallback, expiring semantics, urgent pre-deprecation rule) to cut prose while preserving the policy.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The body is dense with non-obvious operational detail that earns its tokens (provider API quirks, auth requirements, edge cases like ':exacto' and Cloudflare aliasing). However, the long Cloudflare paragraph in particular weaves policy rationale into rambling prose ('Never use LiteLLM's catalog or models.dev...') that could be tightened or moved out of the main flow. | 4 / 5 |
Actionability | Fully executable throughout: exact script invocations with output redirection, env-var sourcing, aws bedrock + jq pipelines, curl + jq for the LiteLLM DB, grep commands for locating and counting edits, plus a concrete example report format. Each command is copy-paste ready and the expected output fields are documented. | 5 / 5 |
Workflow Clarity | Clear five-phase sequence with a checklist, an explicit user-confirmation gate before edits, and a Phase 5 verification step (deprecated=True count). Not 5 because verification lacks an explicit recovery branch — 'Ensure the count matches what you expect' doesn't say what to do on mismatch — and errors/skips are reported but not looped back on. | 4 / 5 |
Progressive Disclosure | Both bundle scripts (extract_models.py, check_provider.py) are real, clearly referenced by path, and their outputs/behaviors are documented inline. The body is a well-structured overview at ~200 lines, but the detailed provider-quirk prose (Cloudflare, Fireworks, OpenRouter suffixes) is reference-grade material inlined in SKILL.md that could live in a separate file. | 4 / 5 |
Total | 17 / 20 Passed |