CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

research

Conduct post-training research for LLMs using the Tinker API — replicate paper results, explore new training ideas, run and monitor experiments, and document findings. Use this skill whenever the user wants to do research, replicate experiments from a paper or repo, investigate training hypotheses, run experiment sweeps, explore post-training techniques (SFT, RL, DPO, distillation, etc.), set up training, write training code, choose a model, tune hyperparameters, manage checkpoints, export weights, or analyze training logs — even if they just say "try this idea" or "let's see what happens if...".

Quality

96%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

SKILL.md
Quality
Evals
Security

Quality

Content

92%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A high-quality, comprehensive skill body: fully executable code across all four training approaches, a clearly sequenced research methodology with explicit validation and monitoring checkpoints, and clean progressive disclosure to 10 verified one-level-deep reference files. The only weakness is mild verbosity in the motivational 'researcher mindset' framing, which could be tightened without losing guidance value.

DimensionReasoningScore

Conciseness

The body is dense and information-driven — tables, complete code, and checklists rather than explanations of known concepts — but the opening researcher-mindset section ("You are always curious... A researcher who doesn't read the literature wastes time rediscovering known results") and repeated exhortations ("always curious" in both the intro and §7, "watch it like a hawk") are trimmable motivational padding, matching the 4 anchor "minor instances of over-explanation that could be trimmed" rather than the 5 anchor where every token earns its place.

4 / 5

Actionability

Copy-paste-ready, complete code for every training approach (SFT, GRPO, DPO, distillation with full configs), eval (run_benchmarks, BenchmarkEvaluator, trajectory analysis), checkpointing, weight export, and CLI commands, covering the common cases — matching the 5 anchor; the rare placeholders ("# sampling_client from tc.save_weights_and_get_sampling_client()") are clearly annotated, not pseudocode.

5 / 5

Workflow Clarity

The 7-step methodology (understand → models → eval first → data → plan → run/monitor → document) is clearly sequenced with explicit validation checkpoints: "Verify config before launching", the 4-step data inspection checklist, baseline eval before training, staged launch ("Start small, scale up"), and a time-phased monitoring loop ("Immediately after launch... First 5-10 steps... Every ~10-20 steps... If anything is off, investigate now") — feedback loops for error recovery exactly as the 5 anchor requires.

5 / 5

Progressive Disclosure

The body is an overview with inline worked examples, and all 10 referenced files (references/sdk.md, models.md, hyperparams.md, cli.md, sft.md, rl.md, preferences.md, distillation.md, ops.md, dev.md) exist, are one level deep with no nested references, and are signaled at point of need ("For the full eval API... read references/ops.md") plus a consolidated index — matching the 5 anchor's well-signaled one-level-deep structure.

5 / 5

Total

19

/

20

Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

An exemplary description: third-person, concrete, comprehensive action coverage, an explicit 'Use when' clause with natural trigger phrases including casual paraphrases, and a clearly pinned niche (Tinker API post-training research) that minimizes conflict risk. It matches the strongest anchor on every dimension.

DimensionReasoningScore

Specificity

The description lists many concrete actions — "replicate paper results, explore new training ideas, run and monitor experiments, and document findings" plus "set up training, write training code, choose a model, tune hyperparameters, manage checkpoints, export weights, or analyze training logs" — comprehensive coverage of the skill's capabilities in third-person voice, matching the 5 anchor; the 4 anchor applies only when there are minor gaps, and none are evident.

5 / 5

Completeness

It explicitly answers both: "what" in the first sentence (concrete post-training research actions via the Tinker API) and "when" via "Use this skill whenever the user wants to..." followed by concrete trigger phrases — the exact structure of the 5-anchor example; score 4 would require the 'when' to be less explicit, which it is not.

5 / 5

Trigger Term Quality

Trigger coverage is comprehensive including synonyms and casual paraphrases: "replicate experiments from a paper or repo", "run experiment sweeps", "SFT, RL, DPO, distillation", "tune hyperparameters", "analyze training logs", and even "try this idea" or "let's see what happens if..." — natural phrases a user would actually say, matching the 5 anchor's requirement of synonyms and variations.

5 / 5

Distinctiveness Conflict Risk

The capability is pinned to a clear niche ("post-training research for LLMs using the Tinker API") with distinct triggers like technique names (SFT, RL, DPO, distillation), checkpoints, and weight export, minimizing conflict with generic ML or document skills; anchor 4's "minor overlap risk" would apply if triggers were broader (e.g., just "do research"), but the trigger list is domain-specific.

5 / 5

Total

20

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

skill_md_line_count

SKILL.md is long (606 lines); consider splitting into references/ and linking

Warning

Total

15

/

16

Passed

Repository

thinking-machines-lab/tinker-cookbook

Path

skills/research/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.