tinker-training-cost

Calculates training costs for Tinker fine-tuning jobs. Use when estimating costs for Tinker LLM training, counting tokens in datasets, or comparing Tinker model training prices. Tokenizes datasets using the correct model tokenizer and provides accurate cost estimates.

Quality

81%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Quality

Content

76%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is highly actionable with executable commands, real pricing/tokenizer tables, and worked examples, and is well structured for navigation. The main gap is the absence of an explicit verification step in the batch tokenization/cost workflow.

Suggestions

Add a verification checkpoint to the workflow, e.g. a step to sanity-check the token count (run --list-models to confirm the model, or compare a small sample's token count against expectations) before trusting the final cost.

Move or clearly label the dated pricing table (Jan 5, 2026) so stale pricing is easy to spot and refresh, instead of embedding it inline as evergreen reference.

Trim a few explanatory asides (e.g. 'different tokenizers produce different token counts for the same text') that restate knowledge Claude already has.

Dimension	Reasoning	Score
Conciseness	The body is mostly lean with executable commands and necessary reference tables, but contains a few explanatory asides Claude already knows and embeds time-sensitive dated pricing inline rather than in a deprecated section.	4 / 5
Actionability	Fully executable, copy-paste-ready commands with real flags (--list-models, --model, --epochs, --json) plus worked cost examples and a present bundled script covering the common cases.	5 / 5
Workflow Clarity	The load-tokenizer → count-tokens → calculate-cost sequence is clearly listed, but this batch dataset operation lacks an explicit validation/verification checkpoint, which caps workflow clarity at 3 per the batch-operation guidance.	3 / 5
Progressive Disclosure	Well-organized single-file overview with clear section headers and a single one-level-deep bundled script reference that exists on disk; no nested references, though some reference tables could arguably live in a separate file.	4 / 5
	Total	16 / 20 Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is strong: it states concrete capabilities, provides an explicit 'Use when' trigger clause, and is scoped to a distinctive niche. Minor gains are available by adding a broader action list and dataset file-extension triggers.

Dimension	Reasoning	Score
Specificity	Names concrete actions ('Calculates training costs', 'Tokenizes datasets using the correct model tokenizer and provides accurate cost estimates') but the list of distinct actions is modest rather than comprehensive, sitting between anchors 3 and 5.	4 / 5
Completeness	Explicitly answers both 'what' (calculates/tokenizes/estimates) and 'when' via a concrete 'Use when ...' clause with multiple specific triggers.	5 / 5
Trigger Term Quality	Includes natural user phrases ('estimating costs for Tinker LLM training', 'counting tokens in datasets', 'comparing Tinker model training prices') but omits synonyms and dataset file extensions like .jsonl.	4 / 5
Distinctiveness Conflict Risk	Targets the distinct Tinker fine-tuning niche with Tinker-specific triggers, giving it a clear niche and minimal conflict risk with unrelated skills.	5 / 5
	Total	18 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/cloud-compute/tinker-training-cost/SKILL.md
Commit: 3a6c3a9

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.