simpo-training

Simple Preference Optimization for LLM alignment. Reference-free alternative to DPO with better performance (+6.4 points on AlpacaEval 2.0). No reference model needed, more efficient than DPO. Use for preference alignment when want simpler, faster training than DPO/PPO.

Quality

79%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./skills/post-training/simpo/SKILL.md

Quality

Content

80%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable with complete configs and commands and uses progressive disclosure well via three real reference files. Its main weakness is workflow clarity: the training workflows lack an explicit validation/evaluation checkpoint, which the rubric caps at 3 for batch operations.

Suggestions

Add an explicit validation/evaluation step to the training workflow (e.g., after launch, check that loss converges and evaluate the resulting model on test_prefs before declaring success) to lift workflow_clarity above the batch-operation cap of 3.

De-duplicate the `accelerate launch` command — show it once and reference the config per workflow — to tighten conciseness.

State at least one concrete post-training verification action (such as running an AlpacaEval 2.0 comparison) so the workflow has a feedback loop rather than only reactive troubleshooting.

Dimension	Reasoning	Score
Conciseness	The body is efficient and avoids explaining concepts Claude already knows, but the `accelerate launch` command is repeated across Quick start and all three workflows and some hyperparameter values reappear in 'Common issues', so minor trimming is possible.	4 / 5
Actionability	Fully executable, copy-paste-ready guidance: complete installation commands, full YAML configs with specific values and annotated ranges, exact launch commands, and three workflows covering the common cases (base model, instruct model, reasoning tasks).	5 / 5
Workflow Clarity	A clear install → configure → launch sequence is present, but training is a long-running batch operation with no explicit validation or evaluation checkpoint before/after launching (e.g., no 'verify loss converges' or 'evaluate on test_prefs' step); per the rubric's batch-operation rule this caps the score at 3.	3 / 5
Progressive Disclosure	Clear overview body with well-signaled, one-level-deep references — loss-functions.md, hyperparameters.md, and datasets.md (all present in references/) are each linked with a descriptive blurb, and detailed material is appropriately split out rather than inlined.	5 / 5
	Total	17 / 20 Passed

Description

78%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description clearly states what SimPO is and gives an explicit 'Use for... when...' trigger, with strong domain keywords. It is somewhat property-heavy rather than action-enumerating, leaving specificity as the weakest dimension.

Dimension	Reasoning	Score
Specificity	Names the domain ('Simple Preference Optimization for LLM alignment') and a couple of concrete actions ('preference alignment', 'training'), but most of the description states properties ('Reference-free alternative', 'No reference model needed', 'more efficient') rather than enumerating actions, so it is not comprehensive.	3 / 5
Completeness	Explicitly answers both what ('Simple Preference Optimization for LLM alignment... No reference model needed') and when ('Use for preference alignment when want simpler, faster training than DPO/PPO') with a concrete 'Use for... when...' trigger clause.	5 / 5
Trigger Term Quality	Good coverage of natural domain terms a practitioner would say — 'preference alignment', 'DPO', 'PPO', 'LLM alignment', 'training', 'reference-free' — though a few natural variants like 'RLHF', 'reward model', or 'fine-tuning' are absent.	4 / 5
Distinctiveness Conflict Risk	SimPO is a distinct named algorithm with clear differentiators ('Reference-free alternative to DPO', 'No reference model needed'), but it still overlaps the broader preference-optimization / DPO alignment niche, so minor conflict risk with closely related skills remains.	4 / 5
	Total	16 / 20 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning
referenced_paths_exist	Referenced path issues: 3 missing	Warning

	Total	14 / 16 Passed

Repository: OpenLAIR/dr-claw
Path: skills/post-training/simpo/SKILL.md
Commit: 4b35756

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.