tinker-fine-tuning

Provides guidance for fine-tuning LLMs using the Tinker cloud training API from Thinking Machines Lab. Use when running supervised fine-tuning, reinforcement learning (GRPO/PPO), or LoRA training on cloud GPUs via Tinker's managed infrastructure instead of local compute.

Quality

89%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

86%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable skill body with excellent progressive disclosure and executable examples. Conciseness and explicit validation feedback loops are the small remaining gaps.

Suggestions

Consolidate the repeated [OPENSCIENCE_USAGE] reporting snippets into a single shared snippet referenced from each workflow to reduce token duplication.

Add an explicit validate->fix->retry checkpoint in the SFT and RL workflows (e.g., verify metrics/KL before saving weights) to strengthen feedback loops for these batch training operations.

Trim explanatory prose in the Cost Estimation section to a terse formula plus the approval gate, since the surrounding tables already convey the detail.

Dimension	Reasoning	Score
Conciseness	Mostly dense, actionable tables and executable code assuming Claude's competence, but the repeated [OPENSCIENCE_USAGE] reporting blocks and some prose restatement add minor tokens that could be trimmed.	4 / 5
Actionability	Provides copy-paste ready executable Python for SFT, RL (cookbook and low-level), DPO CLI commands, plus concrete import lists and hyperparameter tables covering the common cases.	5 / 5
Workflow Clarity	Workflows have explicit checklists and a pre-training cost-approval checkpoint, but the RL/SFT flows lack a hard validate-then-proceed feedback loop beyond KL monitoring, leaving minor validation gaps.	4 / 5
Progressive Disclosure	Clear overview with a Quick Reference table pointing one level deep to 10 reference files (all verified present in ./references/), each well-signaled by topic; content is appropriately split.	5 / 5
	Total	18 / 20 Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, specific description that clearly states both capabilities and trigger conditions with a distinct vendor niche. Minor keyword synonym coverage (e.g., RLHF/alignment) keeps trigger quality just short of perfect.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions — "supervised fine-tuning, reinforcement learning (GRPO/PPO), or LoRA training on cloud GPUs" — naming both the domain and several specific training methods, matching the comprehensive-coverage anchor.	5 / 5
Completeness	Explicitly answers both what ("Provides guidance for fine-tuning LLMs using the Tinker cloud training API") and when ("Use when running supervised fine-tuning, reinforcement learning (GRPO/PPO), or LoRA training...") with concrete trigger phrases.	5 / 5
Trigger Term Quality	Strong natural keywords (fine-tuning, supervised fine-tuning, reinforcement learning, GRPO/PPO, LoRA, cloud GPUs) but misses common synonyms like RLHF/alignment that users might say, placing it just below the comprehensive anchor.	4 / 5
Distinctiveness Conflict Risk	Names a specific vendor API (Tinker, Thinking Machines Lab) and explicitly contrasts "instead of local compute," giving it a clear niche with minimal overlap with other training skills.	5 / 5
	Total	19 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/cloud-compute/tinker/SKILL.md
Commit: 3a6c3a9

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.