simpo-training

Simple Preference Optimization for LLM alignment. Reference-free alternative to DPO with better performance (+6.4 points on AlpacaEval 2.0). No reference model needed, more efficient than DPO. Use for preference alignment when want simpler, faster training than DPO/PPO.

Quality

78%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./06-post-training/simpo/SKILL.md

Quality

Content

72%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable with executable configs and clean commands and uses progressive disclosure well with verified one-level-deep references. Its weaknesses are repeated boilerplate that could be tightened and the absence of explicit validation checkpoints in the training workflows.

Suggestions

Add explicit validation/verification checkpoints to each workflow (e.g. confirm GPU utilization and that loss is decreasing after the first N steps before continuing).

Reduce repeated launch-command and config boilerplate across workflows by factoring common parts into a shared snippet or a base config reference.

Move time-sensitive specifics such as exact library versions and the '+6.4 points' benchmark figure into a dedicated version/results section so the core guidance stays evergreen and lean.

Dimension	Reasoning	Score
Conciseness	Mostly efficient with executable configs and no padding of basic concepts Claude knows, but it repeats launch commands and config boilerplate across three workflows and carries time-sensitive specifics (exact versions, '+6.4 points') that could be tightened, fitting the score-2 anchor rather than the lean score-3 anchor.	2 / 3
Actionability	Provides fully executable install commands, accelerate launch commands, and complete YAML configs with concrete hyperparameter values and inline comments that are copy-paste ready, matching the score-3 anchor and not the incomplete/pseudocode score-2 anchor.	3 / 3
Workflow Clarity	Workflows are clearly sequenced (install -> config -> launch) with symptom->fix issue mappings, but lack explicit validation/verification checkpoints (no loss-curve sanity check, no confirm-training-started step) for a long training job, capping at the score-2 anchor per the rubric's batch-operation guideline rather than the checkpoint-rich score-3 anchor.	2 / 3
Progressive Disclosure	The body is a well-organized overview with three clearly signaled one-level-deep references (loss-functions.md, hyperparameters.md, datasets.md), all confirmed to exist, with detailed math/tuning/dataset content appropriately split out, matching the score-3 anchor rather than the poorly-signaled score-2 anchor.	3 / 3
	Total	10 / 12 Passed

Description

85%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific, complete, and distinctive, clearly stating what SimPO does and when to use it with an explicit trigger clause. Its main weakness is trigger-term phrasing that is slightly awkward and lacks natural keyword variations, capping trigger_term_quality at 2.

Suggestions

Soften awkward trigger phrasing to natural user speech, e.g. 'Use when you want simpler, faster preference alignment than DPO or PPO.'

Add common natural variations of trigger terms (e.g. 'preference learning', 'RLHF alternative', 'reward-free alignment') so it surfaces for a broader range of user phrasings.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete properties/actions ('Reference-free alternative to DPO', 'No reference model needed', 'more efficient than DPO', 'simpler, faster training'), matching the score-3 anchor for multiple specific concrete actions rather than the partial score-2 anchor.	3 / 3
Completeness	Clearly answers both 'what' (a reference-free preference optimization method outperforming DPO) and 'when' via the explicit 'Use for preference alignment when...' trigger clause, satisfying the score-3 anchor for explicit triggers rather than the score-2 'when is missing or only implied' anchor.	3 / 3
Trigger Term Quality	Contains relevant practitioner keywords ('preference alignment', 'DPO', 'PPO', 'reference model') but lacks natural variations and the trigger phrasing 'when want simpler, faster training' is awkward, fitting the score-2 anchor of some relevant keywords with missing common variations rather than the broad score-3 coverage.	2 / 3
Distinctiveness Conflict Risk	Names a distinct algorithm (SimPO) with a well-defined niche as a reference-free DPO alternative and explicit DPO/PPO comparisons, making it unlikely to trigger for unrelated skills per the score-3 anchor rather than the overlapping score-2 anchor.	3 / 3
	Total	11 / 12 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning
referenced_paths_exist	Referenced path issues: 3 missing	Warning

	Total	14 / 16 Passed

Repository: Orchestra-Research/AI-Research-SKILLs
Path: 06-post-training/simpo/SKILL.md
Commit: 773a529

Reviewed: about 12 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.