CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

debug

Diagnose training issues with Tinker — slow steps, hanging sessions, output mismatches, error messages, renderer problems, and deployment issues. Use this skill whenever a user reports that training is slow, steps take too long, sessions are hanging, model outputs differ between Tinker and external engines (vLLM, SGLang), they get a confusing error message, training quality is poor (high KL, bad outputs), or they suspect something is wrong. Also trigger when users ask "is this a Tinker issue or my issue?", "is Tinker down?", report unexpected wait times, see output quality regressions, get opaque errors, or want to profile/debug their training or deployment pipeline. This skill walks through systematic triage to determine root cause.

Quality

94%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A strong diagnostic skill body: five well-sequenced triage paths with validation routing, escalation checklists, and genuinely executable diagnostics, with extended detail correctly pushed into five real reference files. The main defects are redundancy — a verbatim-duplicated decision-tree block, repeated pyinstrument instructions, and a symptom table that restates earlier fixes — and one reference (serialization-test.md) not linked from its most relevant section.

Suggestions

Delete the duplicated decision-tree block (the five repeated lines under "Output differs between Tinker and external engine") and merge the repeated pyinstrument install/invocation snippet in Step 4 Option B into the Step 3 mention.

Link `references/serialization-test.md` from Step 1's known-problem-versions section (pydantic beta regression), where the benchmark is directly actionable, not just from the end-of-file reference list.

Consider trimming the "Common resolutions" table to symptoms not already covered by the triage sections, or moving it into a reference file, to cut duplicated content.

DimensionReasoningScore

Conciseness

The body is largely lean and assumes competence — it teaches only Tinker-specific knowledge (SDK threading model, known-bad versions, GIL failure modes) without explaining general concepts. However, there is real waste: the "Output differs between Tinker and external engine" decision-tree block is duplicated verbatim (lines 571–581 repeat 571–576), the pyinstrument install/invocation appears twice (Step 3 and Step 4 Option B), and the "Common resolutions" table substantially restates fixes already given in the triage sections. That is more than the 'every token earns its place' bar of level 5, though the waste is bounded, keeping it above level 3's 'some unnecessary explanation'.

4 / 5

Actionability

Guidance is overwhelmingly executable and copy-paste ready: environment version checks, the token-comparison script with first-divergence reporting, the GIL monitor thread, the service smoke test, `pyinstrument --async-mode=enabled`, `TINKER_SUBPROCESS_SAMPLING=1`, and specific version pins (`pydantic<2.13`, `transformers>=5.0`). Specific examples cover the common cases across all five triage paths, matching the level-5 anchor; the only slightly illustrative snippet (the `build_supervised_example` hasattr loop) is too minor to drop it to level 4.

5 / 5

Workflow Clarity

Each triage path is a numbered, ordered sequence with explicit validation checkpoints and feedback routing — e.g., "High submit time → ... Go to Step 4", "If tokens match: The renderer is correct. The issue is elsewhere", smoke-test interpretation branching, and a five-condition escalation checklist. "Work through these steps in order. Most issues are caught in steps 1-3" sets the entry sequence. This matches the level-5 anchor of clear sequencing, explicit validation, and error-recovery loops.

5 / 5

Progressive Disclosure

Structure is good: the SKILL.md body carries the core triage flows while all five real reference files (verified to exist and match their claimed purpose) hold the deep material, cited inline at the point of need ("read references/async-task-dump.md for ready-to-paste diagnostic code") and again in a final list — one level deep, no nesting. Two gaps keep it below level 5: `references/serialization-test.md` is only surfaced in the end-of-file list rather than from Step 1's pydantic section where it is directly relevant, and the duplicated decision-tree block plus some inlined tables suggest content that could be split more aggressively.

4 / 5

Total

18

/

20

Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

An exemplary description: it states concrete capabilities that mirror the body's structure, provides dense natural-language trigger phrases with synonym coverage, and defines a distinct Tinker-specific niche. Third-person voice is used consistently; the only nitpick is mild redundancy between the two trigger clauses ("confusing error message" / "opaque errors", "training is slow" / "unexpected wait times"), which serves trigger breadth rather than padding.

DimensionReasoningScore

Specificity

The description names the domain ("Diagnose training issues with Tinker") and enumerates comprehensive concrete capabilities: "slow steps, hanging sessions, output mismatches, error messages, renderer problems, and deployment issues", plus "systematic triage to determine root cause" and "profile/debug their training or deployment pipeline". These map one-to-one onto the skill's five triage paths, with no gaps — matching the level-5 anchor of multiple specific concrete actions with comprehensive coverage, not the level-4 anchor of minor coverage gaps.

5 / 5

Completeness

Both questions are answered explicitly and concretely: the "what" opens the description ("Diagnose training issues with Tinker — slow steps, ... deployment issues") and the "when" is given twice ("Use this skill whenever a user reports that..." and "Also trigger when users ask..."). This matches the level-5 anchor ("clearly and explicitly answers both what AND when with concrete trigger phrases"), not level 4 where the 'when' could be more explicit.

5 / 5

Trigger Term Quality

Trigger phrases are exactly what affected users would type: "training is slow", "steps take too long", "sessions are hanging", "is this a Tinker issue or my issue?", "is Tinker down?", "output quality regressions", "unexpected wait times", "opaque errors". Synonym variants are covered ("confusing error message" / "opaque errors"), matching the level-5 anchor of comprehensive natural terms including synonyms rather than the level-4 anchor of a few missing terms.

5 / 5

Distinctiveness Conflict Risk

The skill occupies a clear niche — Tinker-specific training debugging — with distinct triggers ("is Tinker down?", "differ between Tinker and external engines (vLLM, SGLang)", "high KL") that would not fire for generic debugging or coding skills. Minimal conflict risk, matching the level-5 anchor of a clear niche with distinct triggers.

5 / 5

Total

20

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

skill_md_line_count

SKILL.md is long (657 lines); consider splitting into references/ and linking

Warning

Total

15

/

16

Passed

Repository

thinking-machines-lab/tinker-cookbook

Path

skills/debug/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.