CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

tinker-fine-tuning

Provides guidance for fine-tuning LLMs using the Tinker cloud training API from Thinking Machines Lab. Use when running supervised fine-tuning, reinforcement learning (GRPO/PPO), or LoRA training on cloud GPUs via Tinker's managed infrastructure instead of local compute.

Quality

85%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

78%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable skill body: clear use/don't-use scoping, three fully worked workflows with checklists, executable code, concrete hyperparameter and pricing tables, and exemplary progressive disclosure via a verified Quick Reference table. Weaknesses are minor: repeated usage-reporting boilerplate, a few undefined variables in the low-level RL example, and validation steps that are listed but not wired into explicit validate-and-retry loops.

Suggestions

Collapse the three inline [OPENSCIENCE_USAGE] code blocks into the single 'Automatic Usage Reporting' section and reference it from the workflows ('add the usage-report line from the Usage Tracking section') — this removes ~40 duplicated lines and lifts conciseness toward anchor 5.

Make the low-level RL example self-contained or explicitly framed: define (or link to) 'dataset', 'renderer', and 'compute_reward' — e.g., 'renderer = get_renderer(get_recommended_renderer_name(model_name))' — so the example is copy-paste executable rather than having undefined names.

Add explicit validate-and-retry checkpoints to the training workflows: after saving weights, verify by sampling a few completions from the saved model path; when KL exceeds 0.01 or reward stays at 0, inline the remediation steps (reduce LR / increase group size) directly in the workflow rather than only in the Common Issues table.

DimensionReasoningScore

Conciseness

The body is dense with genuinely non-obvious, API-specific material (hyperparameter tables, model pricing, LoRA LR caveats, 'cosine does NOT work') and assumes Claude's competence — no padding explaining what fine-tuning is. It falls short of anchor 5 because the '[OPENSCIENCE_USAGE]' boilerplate is repeated verbatim three times in code examples plus a dedicated section (~40 redundant lines), and installation/API-key guidance is duplicated between the Installation section and the Common Issues table — 'minor instances of over-explanation that could be trimmed' matches anchor 4 exactly.

4 / 5

Actionability

Mostly executable, copy-paste-ready guidance: complete Cookbook SFT and RL scripts with real imports and configs, a runnable DPO CLI command, install commands, and save/resume snippets. Not anchor 5 because the low-level RL example references undefined variables ('dataset', 'renderer', 'compute_reward') without definition or pointer, and the Evaluations inline snippet is a fragment ('...' in the blueprint apply) — these are 'concrete code or commands with minor gaps' (anchor 4) rather than fully self-contained coverage.

4 / 5

Workflow Clarity

Each workflow has an explicit sequenced checklist (prepare data → choose model → set hyperparameters → run → monitor metrics → save), monitoring metrics with thresholds ('Keep KL below 0.01'), and a pre-training cost-estimation approval gate ('Present the cost estimate to the user for approval before starting training'), which satisfies the batch-operation validation requirement. It sits below anchor 5 because checkpoints are advisory rather than explicit validate-and-retry loops — e.g., no step to verify the saved sampler works or to abort training on diverging metrics, and remediation (KL, reward-stuck) lives in a separate Common Issues table rather than inline feedback loops — matching anchor 4 ('Clear sequence with most checkpoints present; minor validation gaps').

4 / 5

Progressive Disclosure

The body is a genuine overview — 'When to Use', a comparison table, Quick Reference, core workflow code, and hyperparameter tables — with a 'Quick Reference' table pointing to ten clearly-labeled reference files, all of which exist on disk (verified), one level deep, with no nested references. This matches anchor 5 ('Clear overview with well-signaled one-level-deep references; content appropriately split; easy navigation'); anchor 4 would require organization gaps or buried references, which are absent.

5 / 5

Total

17

/

20

Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description in third-person voice with an explicit 'Use when' trigger clause, concrete capability enumeration, and a clear niche that distinguishes Tinker cloud training from local fine-tuning alternatives. The only improvements are minor: adding common synonyms (SFT, RLHF, DPO) and slightly more concrete action verbs than 'Provides guidance'.

DimensionReasoningScore

Specificity

The description names the domain ('fine-tuning LLMs using the Tinker cloud training API') and lists several concrete capabilities: 'supervised fine-tuning, reinforcement learning (GRPO/PPO), or LoRA training on cloud GPUs'. It falls just short of anchor 5 because the primary action 'Provides guidance' is generic and coverage of adjacent capabilities (DPO, evaluation, vision-language models) present in the body is omitted — a minor gap, matching anchor 4 ('Lists several specific actions; minor gaps in coverage') rather than 3, which requires only 1-2 concrete actions.

4 / 5

Completeness

It explicitly answers both questions: 'what' — 'Provides guidance for fine-tuning LLMs using the Tinker cloud training API from Thinking Machines Lab' — and 'when' — 'Use when running supervised fine-tuning, reinforcement learning (GRPO/PPO), or LoRA training on cloud GPUs via Tinker's managed infrastructure instead of local compute.' This matches the anchor-5 example pattern (concrete what + explicit 'Use when' clause with concrete triggers); anchor 4 would require the 'when' to be less explicit or specific.

5 / 5

Trigger Term Quality

Natural trigger terms users would say are present: 'fine-tuning', 'supervised fine-tuning', 'reinforcement learning (GRPO/PPO)', 'LoRA training', 'cloud GPUs'. It is not anchor 5 because common synonyms users might naturally say — 'SFT', 'RLHF', 'DPO', 'preference training', 'train a model' — are missing; it is clearly above anchor 3, which expects missing common variations.

4 / 5

Distinctiveness Conflict Risk

The description is anchored to a uniquely named service ('Tinker cloud training API from Thinking Machines Lab') with a clear niche and an explicit disambiguator ('instead of local compute'), so it is unlikely to trigger for local fine-tuning skills (Axolotl, Unsloth, TRL). This matches anchor 5 ('Clear niche with distinct triggers; minimal conflict risk'); anchor 4 would imply some overlap with closely related skills, which the 'instead of local compute' framing largely rules out.

5 / 5

Total

18

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository

synthetic-sciences/openscience

Path

backend/cli/skills/cloud-compute/tinker/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.