hugging-face-model-trainer

This skill should be used when users want to train or fine-tune language models using TRL (Transformer Reinforcement Learning) on Hugging Face Jobs infrastructure. Covers SFT, DPO, GRPO and reward modeling training methods, plus GGUF conversion for local deployment. Includes guidance on the TRL Jobs package, UV scripts with PEP 723 format, dataset preparation and validation, hardware selection, cost estimation, Trackio monitoring, Hub authentication, and model persistence. Should be invoked for tasks involving cloud GPU training, GGUF conversion, or when users mention training on Hugging Face Jobs without local GPU setup.

Quality

88%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

Highly actionable content with strong, validated workflows and well-organized one-level references to real bundle files. The main weakness is conciseness: key directives are restated 3-4 times across sections and some inline content overlaps with the reference files.

Suggestions

Consolidate the repeated timeout, push-to-Hub, and submit-inline directives into their canonical sections (Timeout Management, Critical: Saving Results, Key Directives) and reference them elsewhere instead of restating; remove the 'What is GGUF' concept explainer since Claude already knows the format.

Move the Common Failure Modes detail into references/troubleshooting.md (or vice versa) to eliminate the duplicated troubleshooting coverage and shorten the body.

Either reference references/reliability_principles.md from the body or drop it from the bundle, so every bundled reference is reachable via clear navigation.

Dimension	Reasoning	Score
Conciseness	Mostly actionable operational content, but the same directives are repeated across sections (30-min timeout warning in four places, push-to-Hub/ephemeral warning in four places, 'submit inline via hf_jobs()' in four places) and the 'What is GGUF' block explains a concept Claude already knows, so it could be tightened rather than rated efficient.	3 / 5
Actionability	Provides fully executable, copy-paste-ready guidance for the common cases — a complete PEP 723 SFT script with real model/dataset names and full SFTConfig, exact `hf jobs` CLI flag ordering, the cost-estimation command, and a dataset-validation workflow with mapping code — meeting the 'specific examples cover the common cases' anchor.	5 / 5
Workflow Clarity	Multi-step processes are clearly sequenced with explicit validation checkpoints and feedback loops — the dataset workflow (inspect → check ✓ READY / ✗ NEEDS MAPPING markers → apply mapping → launch) validates before expensive GPU training, plus Prerequisites and Hub-push Verification checklists and ordered failure-mode fixes, satisfying the checklists/feedback-loop anchor.	5 / 5
Progressive Disclosure	Good one-level-deep structure with clearly signaled references to real files (training_methods.md, hardware_guide.md, gguf_conversion.md, trackio_guide.md, hub_saving.md, troubleshooting.md, training_patterns.md) and a Resources index, but the ~700-line body inlines content that could live in references (the Common Failure Modes vs Troubleshooting overlap, full multi-approach examples) and leaves reliability_principles.md unreferenced, so it is not the cleanly split overview of a 5.	4 / 5
	Total	17 / 20 Passed

Description

96%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description that explicitly states capabilities and trigger conditions with concrete, natural phrasing and synonyms. The only soft spot is slight breadth in the generic training terms that could let it overlap with a general fine-tuning skill.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions and methods — 'SFT, DPO, GRPO and reward modeling', 'GGUF conversion', 'dataset preparation and validation', 'hardware selection', 'cost estimation', 'Trackio monitoring', 'Hub authentication, and model persistence' — giving comprehensive coverage rather than the minor gaps of a 4.	5 / 5
Completeness	Explicitly answers both what (train/fine-tune with TRL methods + GGUF conversion) and when ('This skill should be used when...', 'Should be invoked for tasks involving cloud GPU training, GGUF conversion, or when users mention training on Hugging Face Jobs'), matching the concrete-trigger-phrases anchor.	5 / 5
Trigger Term Quality	Natural user phrasing is well covered with synonyms and a format token — 'train or fine-tune language models', 'GGUF conversion', 'cloud GPU training', 'training on Hugging Face Jobs' — so it is not merely the 'good coverage, a few missing' of a 4.	5 / 5
Distinctiveness Conflict Risk	The TRL-on-Hugging-Face-Jobs + GGUF niche is mostly distinct with minimal conflict, but 'train or fine-tune language models' and 'cloud GPU training' are broad enough to risk minor overlap with a generic fine-tuning skill, so it sits at 'mostly distinct' rather than the clear-niche of a 5.	4 / 5
	Total	19 / 20 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
skill_md_line_count	SKILL.md is long (724 lines); consider splitting into references/ and linking	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	14 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/ml-training/hugging-face-model-trainer/SKILL.md
Commit: 1d182e9

Reviewed: 3 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.