finetuning-method-selection

Decide whether to fine-tune at all, and route to the right method (SFT, DPO/ORPO/KTO, GRPO/RLVR, continued pretraining) and base model. Use when starting any fine-tuning effort, when unsure whether RAG or prompting would suffice, or when choosing between preference-optimization and reinforcement methods.

Quality

89%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

78%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured router skill that gives concrete, actionable decision rules and defers execution detail to appropriately-linked reference and sibling-skill files. Main improvement opportunity is reducing redundancy between the Quick Reference table and the Method Router tree.

Suggestions

Consolidate the Quick Reference table and the Method Router code block, which encode largely the same routing logic — keep one as the canonical tree and have the other cross-reference it rather than restating it.

The Key Routing Facts, Worked Routing Examples, and Common Routing Mistakes sections reinforce overlapping points; consider merging mistakes into the corresponding worked example or routing fact to cut repetition.

Dimension	Reasoning	Score
Conciseness	Largely lean and assumes Claude's competence (no primers on what SFT/RAG are), but the Quick Reference table and the Method Router code block encode overlapping routing logic that could be consolidated.	4 / 5
Actionability	Provides concrete, executable routing rules — data-shape-to-method mapping, explicit volume thresholds (10MB/500MB/10GB), CPT LR ≈ 10% of pretrain LR, and a memory formula — with minor execution detail deferred to downstream skills by design.	4 / 5
Workflow Clarity	Clear sequence from Off-Ramps First through Method Router, Model Selection, and Memory Feasibility, with an explicit eval-harness "Stop" gate; minor checkpoint gaps but no destructive/batch cap applies.	4 / 5
Progressive Disclosure	Acts as an overview pointing to verified one-level-deep references (references/model-catalog.md, references/memory-math.md, both present) and clearly-signaled handoffs to downstream skills, with detailed material appropriately split out.	5 / 5
	Total	17 / 20 Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, specific description that clearly states both the capability and explicit trigger conditions, naming the full method set and base-model routing. It is distinguishable from sibling skills and free of vague fluff.

Dimension	Reasoning	Score
Specificity	Names multiple concrete routing actions and enumerates the full method set ("route to the right method (SFT, DPO/ORPO/KTO, GRPO/RLVR, continued pretraining) and base model"), giving comprehensive coverage of what the skill does.	5 / 5
Completeness	Explicitly answers both what ("Decide whether to fine-tune at all, and route to the right method... and base model") and when ("Use when starting any fine-tuning effort, when unsure whether RAG or prompting would suffice, or when choosing...") with concrete trigger phrases.	5 / 5
Trigger Term Quality	Includes natural terms a user would actually say — "fine-tuning effort", "RAG or prompting", "preference-optimization and reinforcement methods" — with synonyms and method acronyms covering the common phrasings.	5 / 5
Distinctiveness Conflict Risk	Occupies a clear router niche (decide-whether-and-which-method) distinct from the downstream execution skills it names, with triggers unlikely to fire for the wrong skill.	5 / 5
	Total	20 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: wshobson/agents
Path: plugins/llm-finetuning/skills/finetuning-method-selection/SKILL.md
Commit: 4236bb9

Reviewed: 6 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.