grpo-rl-training

Expert guidance for GRPO/RL fine-tuning with TRL for reasoning and task-specific model training

Quality

58%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Critical

Do not install without reviewing

Fix and improve this skill with Tessl

tessl review fix ./backend/cli/skills/ml-training/grpo-rl-training/SKILL.md

Quality

Content

63%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is rich with executable, expert-grade GRPO/TRL guidance and a clear workflow, but it is verbose for its stated expert audience and ships as a monolith with broken references to nonexistent templates/ and examples/ directories. Splitting reference material into real bundle files and trimming restated fundamentals would materially improve it.

Suggestions

Create the referenced templates/ and examples/ directories (or remove the broken references) and move the bulk of reward-function examples and full config blocks there, keeping SKILL.md as a lean overview.

Trim expert-redundant prose — the 'Mathematical Intuition' restatement of GRPO and generic 'Pro Tips' — since the skill is explicitly for expert-level implementation.

Define or import the undefined helpers (extract_answer, extract_final_answer, compute_score) so the code examples are fully runnable as written.

Dimension	Reasoning	Score
Conciseness	The ~560-line body is mostly useful and concrete, but includes padded explanatory prose an expert audience does not need ('Mathematical Intuition' restating GRPO basics, 'Critical Insight', 'Pro Tips', a full best-practices checklist and troubleshooting guide) that could be tightened.	3 / 5
Actionability	It provides mostly executable, copy-paste-ready code (full GRPOConfig blocks, reward-function templates, trainer setup), but relies on several undefined helpers (extract_answer, extract_final_answer, compute_score, CUSTOM_SYSTEM_PROMPT) and ends the Unsloth example with a GRPOTrainer(...) placeholder, leaving minor gaps.	4 / 5
Workflow Clarity	The four-step Implementation Workflow (Dataset → Reward → Config → Setup/Training) is clearly sequenced with before/during/after checklists and monitoring guidance (sample generations every 50-100 steps, validate on holdout set), satisfying the batch-operation validation expectation; only minor checkpoint explicitness is missing.	4 / 5
Progressive Disclosure	Section headers give reasonable structure, but the file is a monolithic 560-line document inlining content that belongs in separate files (full configs, multiple reward examples, troubleshooting), and it references templates/ and examples/ directories that do not exist in the bundle.	3 / 5
	Total	14 / 20 Passed

Description

53%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description names a specific, distinct niche but reads as a capability tag rather than trigger guidance: it states what the skill does without saying when Claude should invoke it. Adding an explicit 'Use when...' clause with natural user phrases would lift completeness and trigger-term quality.

Suggestions

Add an explicit trigger clause, e.g. 'Use when fine-tuning a model with reinforcement learning via GRPO/TRL, designing custom reward functions, or enforcing structured output formats during RL training.'

Include natural user-facing synonyms ('reinforcement learning training', 'RLHF', 'reward modeling') rather than only technical jargon.

Mention 1-2 more concrete capabilities from the body (reward function design, multi-stage training) to broaden specificity.

Dimension	Reasoning	Score
Specificity	The phrase 'GRPO/RL fine-tuning with TRL for reasoning and task-specific model training' names the domain and a couple of concrete actions (fine-tuning, task-specific training), but is not comprehensive — it omits reward function design, format enforcement, and other capabilities the body actually covers.	3 / 5
Completeness	It gives a clear 'what' (GRPO/RL fine-tuning with TRL) but no 'when' — there is no 'Use when...' clause or explicit trigger guidance, which per the rubric caps completeness at 3.	3 / 5
Trigger Term Quality	It surfaces relevant terms a user might say ('GRPO', 'RL fine-tuning', 'TRL', 'reasoning'), but these skew technical and miss common natural variations/synonyms a user would actually utter like 'reinforcement learning training', 'RLHF', 'reward model', or 'DPO'.	3 / 5
Distinctiveness Conflict Risk	The GRPO/RL-with-TRL niche is mostly distinct with minimal conflict risk, though it could overlap slightly with broader SFT/DPO/RLHF fine-tuning skills.	4 / 5
	Total	13 / 20 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
skill_md_line_count	SKILL.md is long (574 lines); consider splitting into references/ and linking	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	14 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/ml-training/grpo-rl-training/SKILL.md
Commit: fcd2e11

Reviewed: 2 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.