hugging-face-model-trainer

This skill should be used when users want to train or fine-tune language models using TRL (Transformer Reinforcement Learning) on Hugging Face Jobs infrastructure. Covers SFT, DPO, GRPO and reward modeling training methods, plus GGUF conversion for local deployment. Includes guidance on the TRL Jobs package, UV scripts with PEP 723 format, dataset preparation and validation, hardware selection, cost estimation, Trackio monitoring, Hub authentication, and model persistence. Should be invoked for tasks involving cloud GPU training, GGUF conversion, or when users mention training on Hugging Face Jobs without local GPU setup.

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is highly actionable with executable examples and a clearly sequenced, validation-backed workflow, supported by a well-organized reference bundle whose files all exist. Its main weakness is verbosity from restating the same directives across multiple sections.

Suggestions

Consolidate the repeated guidance about submitting scripts inline, avoiding polling, setting timeouts, and pushing to Hub so it appears once (e.g., in Key Directives) rather than being restated in Asynchronous Job Guidelines and Key Takeaways.

Trim the meandering Trackio defaults prose into a tighter checklist or move the extended configuration detail into references/trackio_guide.md.

Consider moving the full hardware and timeout tables into references/hardware_guide.md, keeping only a compact summary inline, to push progressive_disclosure toward a leaner overview.

Dimension	Reasoning	Score
Conciseness	Mostly efficient and avoids explaining concepts Claude already knows, but restates the same directives across Key Directives, Asynchronous Job Guidelines, and Key Takeaways (submit inline, don't poll, set timeout, push to Hub), and the Trackio defaults section meanders, so it could be tightened.	3 / 5
Actionability	Provides fully executable inline hf_jobs('uv', {...}) examples, complete CLI syntax with correct/wrong comparisons, copy-paste DPO mapping code, and concrete config snippets covering the common training cases.	5 / 5
Workflow Clarity	Sequences the job lifecycle clearly (create script, submit inline, report details, wait for user) with explicit validation checkpoints (validate datasets before GPU training), verification checklists, and feedback loops in the failure-mode section.	5 / 5
Progressive Disclosure	Body is an overview with well-signaled one-level-deep references to references/.md and scripts/.py (all verified to exist) plus a consolidated Resources section, but the SKILL.md itself inlines substantial detail (hardware tables, timeout tables, failure modes) that keeps it just short of a lean pure-overview split.	4 / 5
	Total	17 / 20 Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is comprehensive and well-structured, explicitly covering both what the skill does and when to invoke it with concrete trigger phrases. Its only notable weakness is slightly incomplete keyword/synonym coverage (missing LoRA, RLHF, and common alternative phrasings).

Dimension	Reasoning	Score
Specificity	Lists multiple concrete capabilities — SFT, DPO, GRPO, reward modeling, GGUF conversion, dataset validation, hardware selection, cost estimation, Trackio monitoring, Hub auth, model persistence — providing comprehensive coverage rather than generic language.	5 / 5
Completeness	Explicitly answers both what (TRL training methods, GGUF conversion, supporting guidance areas) and when ('Should be invoked for tasks involving cloud GPU training, GGUF conversion, or when users mention training on Hugging Face Jobs'), with concrete trigger phrases.	5 / 5
Trigger Term Quality	Includes natural phrases users would say ('train or fine-tune language models', 'GGUF conversion', 'training on Hugging Face Jobs', 'cloud GPU training'), but misses common synonyms like LoRA, RLHF, and several alternative phrasings users might naturally use.	4 / 5
Distinctiveness Conflict Risk	Carves a clear niche — TRL on Hugging Face Jobs cloud GPUs with GGUF conversion — with distinct triggers and minimal overlap risk against other skills.	5 / 5
	Total	19 / 20 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
skill_md_line_count	SKILL.md is long (724 lines); consider splitting into references/ and linking	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	14 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/ml-training/hugging-face-model-trainer/SKILL.md
Commit: 3a6c3a9

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.