grpo-rl-training

Expert guidance for GRPO/RL fine-tuning with TRL for reasoning and task-specific model training

Quality

57%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Critical

Do not install without reviewing

Fix and improve this skill with Tessl

tessl review fix ./06-post-training/grpo-rl-training/SKILL.md

Quality

Content

65%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is highly actionable with extensive executable code and a clear staged workflow, but it is over-long with some prose/table redundancy, relies on advisory rather than enforced validation checkpoints for risky training, and references bundle directories (templates/, examples/) that do not exist. Tightening the prose and either providing or removing the broken references would improve it.

Suggestions

Remove or actually create the referenced 'templates/' and 'examples/' directories so navigation is not broken, or drop those mentions from the Usage Instructions section.

Tighten redundant content: keep either the 'Critical Hyperparameters' table or the inline prose advice, and trim restated agent usage instructions that duplicate the workflow.

Add an explicit validate-then-proceed feedback loop around the training step (e.g. run a short reward-function smoke test, confirm reward_std > 0 before launching the full run, and define a rollback/early-stop condition) to raise workflow clarity.

Dimension	Reasoning	Score
Conciseness	The body is mostly dense and earns its place with executable examples, but it repeats information between prose and tables (e.g. hyperparameter guidance in both 'Critical Hyperparameters' table and surrounding text) and includes light padding Claude already knows ('garbage in = garbage out', restated 'Usage Instructions for Agents'), so it could be tightened rather than being fully lean.	2 / 3
Actionability	It provides numerous complete, copy-paste-ready Python blocks (dataset prep, reward functions, GRPOConfig, GRPOTrainer setup, Unsloth setup, LoRA merge, inference) plus concrete hyperparameter tables, matching the score-3 anchor for fully executable guidance; only a couple of examples use '...' placeholders.	3 / 3
Workflow Clarity	The Implementation Workflow (Steps 1-4) is clearly sequenced and backed by Before/During/After checklists and monitoring warning signs, but for a multi-hour, hard-to-undo training run the validation is advisory (checklists and metrics to watch) rather than explicit enforced validate-fix-retry feedback loops, so it sits at the score-2 anchor rather than 3.	2 / 3
Progressive Disclosure	The body has clear internal sectioning, but no bundle files exist (references/, scripts/, assets/ absent) while the text references nonexistent 'templates/' and 'examples/' directories, and the document is a ~560-line monolith with inline content that would benefit from being split out; structure is present but navigation is weak/broken.	2 / 3
	Total	9 / 12 Passed

Description

50%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is a solid one-liner that names the technique and library, but it omits an explicit 'Use when' trigger and lists few concrete actions, leaving both completeness and trigger coverage at the midpoint. Adding a natural-language trigger clause and one or two more concrete capabilities would lift it across dimensions.

Suggestions

Add an explicit trigger clause, e.g. 'Use when fine-tuning a model with GRPO/RL, designing reward functions, or enforcing structured output formats via TRL'.

Name a few concrete actions (e.g. 'design reward functions, configure GRPO training, track reward/KL metrics, merge LoRA adapters') to raise specificity from 2 to 3.

Include common natural trigger variations users would say (RLHF, reinforcement learning training, reward modeling, structured-output fine-tuning) to broaden trigger term coverage.

Dimension	Reasoning	Score
Specificity	The description names the domain and tool ('GRPO/RL fine-tuning with TRL') and a partial purpose ('reasoning and task-specific model training'), but it does not list multiple distinct concrete actions the way the score-3 anchor does ('Extract... fill forms... merge documents'). It is not vague enough for a 1, yet lacks the comprehensive action list needed for a 3.	2 / 3
Completeness	It states what the skill does but provides no 'Use when...' clause or equivalent explicit trigger guidance; per the rubric guideline, a missing trigger clause caps completeness at 2, and it is above 1 because the 'what' is clearly stated.	2 / 3
Trigger Term Quality	It surfaces real relevant terms an ML practitioner would say ('GRPO', 'RL fine-tuning', 'TRL', 'reasoning'), but omits common natural variations users might voice ('RLHF', 'reward modeling', 'reinforcement learning training', 'reward functions'), so coverage is partial rather than broad.	2 / 3
Distinctiveness Conflict Risk	'GRPO/RL fine-tuning with TRL' is a reasonably distinct niche unlikely to trigger unrelated skills, but it could still overlap with generic SFT/DPO/PPO fine-tuning skills and, without an explicit trigger clause, the boundary is only implied rather than crisp.	2 / 3
	Total	8 / 12 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
skill_md_line_count	SKILL.md is long (573 lines); consider splitting into references/ and linking	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	14 / 16 Passed

Repository: Orchestra-Research/AI-Research-SKILLs
Path: 06-post-training/grpo-rl-training/SKILL.md
Commit: 773a529

Reviewed: about 13 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.