CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

model-economics

Cost modeling and ROI analysis for specialized LLM development. Use when deciding whether to train a custom model, estimating total cost, or calculating break-even vs frontier APIs. Covers training costs, inference costs, and time-to-ROI projections.

66

Quality

83%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

SKILL.md
Quality
Evals
Security

Quality

Content

71%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable — complete, runnable Python for the two core calculations, concrete cost tables, dollar-threshold decision criteria, and diagnostic questions — with a coherent logical flow and a built-in viability check. Its weaknesses are token efficiency and structure: large dated pricing/throughput reference tables are inlined in SKILL.md rather than split into clearly signaled reference files, which both bloats the overview and guarantees staleness as prices change.

Suggestions

Move the frontier pricing tiers, GPU cost tables, and vLLM throughput benchmarks into a references/ file (e.g. references/pricing.md) with a clearly signaled pointer, keeping SKILL.md to the decision framework and calculations — this would improve both conciseness and progressive disclosure.

Place time-sensitive model versions and prices under an explicitly dated section (or 'verify current pricing via websearch' wrapper) so stale entries are quarantined rather than scattered through decision-critical tables.

Number the implicit assessment workflow explicitly (gather the four Quick Assessment inputs → estimate training cost → compute inference cost → run break-even analysis → apply the decision framework) so the multi-step sequence and its checkpoints are unambiguous.

DimensionReasoningScore

Conciseness

The body is mostly efficient — dense tables and code rather than explanations of concepts Claude already knows — but it inlines large time-sensitive reference tables ('Reference tiers as of early 2026', model-specific prices like 'GPT-5.2 Pro | $21.00 | $168.00', per-GPU throughput rates) that will rot and belong in a dated reference file. The rubric explicitly penalizes time-sensitive version/price data outside a deprecated/old-patterns section, keeping this below the 'minor instances of over-explanation' of a 4.

3 / 5

Actionability

The guidance is fully executable: copy-paste-ready Python functions (`inference_cost_per_million_tokens`, `break_even_analysis` with complete argument handling and a non-viable guard), a worked example with concrete inputs ($5K/mo spend, $500 training, $200/mo inference) and expected output, plus dollar-threshold decision criteria ('Monthly API spend > $1,000'). Specific examples cover the common cases, matching the score-5 anchor. (Minor nit: the example's stated '~1.1 months' break-even is actually ~0.9 by the formula, but the code itself is correct and runnable.)

5 / 5

Workflow Clarity

The sequence is clear and logically ordered — 'Use this skill as the **first step**', then pricing context → training cost → inference cost → break-even → decision framework → 'Quick Assessment Questions' to gather inputs — and the break-even function embeds a viability checkpoint ('monthly_savings <= 0' → 'viable: False, reason: ...'). It is not a 5 because the end-to-end procedure is implied by section order rather than explicitly enumerated, and there is no explicit validation/retry loop on the recommendation itself; it is above 3 because most checkpoints (the viability guard, the decision thresholds, the assessment questions) are present and concrete.

4 / 5

Progressive Disclosure

There are no bundle files (no references/, scripts/, or assets/), and the skill is a ~228-line monolith. Sections are well-organized with headers, and the one external pointer ('Load `tinker-training-cost` skill for exact per-token rates') is clearly signaled, but content that would sit better in a separate reference file — the frontier pricing tables, GPU price lists, and throughput benchmarks — is inlined in SKILL.md. This matches the score-3 anchor ('content that should be separate is inline') more than the appropriately-split structure of a 4.

3 / 5

Total

15

/

20

Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it clearly and explicitly states both what the skill does and when to use it, with concrete, domain-specific action verbs and trigger phrases in third person. The only weakness is moderate keyword coverage — it omits the very common 'fine-tune'/'fine-tuning' vocabulary users would naturally use, which slightly narrows trigger matching and leaves room for overlap with neighboring fine-tuning skills.

Suggestions

Add the natural synonyms users actually say for this task — e.g. 'fine-tuning cost', 'should I fine-tune or use an API', 'custom model vs frontier API' — to broaden trigger-term coverage.

Sharpen distinctiveness from adjacent fine-tuning skills by anchoring triggers on the economic decision (e.g. 'Use when evaluating whether the economics justify training') rather than on training costs generally.

DimensionReasoningScore

Specificity

The description lists multiple concrete actions — 'Cost modeling and ROI analysis', 'deciding whether to train a custom model, estimating total cost, or calculating break-even vs frontier APIs' — and enumerates its coverage ('training costs, inference costs, and time-to-ROI projections'). Coverage is comprehensive for the skill's domain; unlike the score-4 anchor there are no meaningful gaps in the action list.

5 / 5

Completeness

It explicitly answers both parts: the 'what' ('Cost modeling and ROI analysis for specialized LLM development... Covers training costs, inference costs, and time-to-ROI projections') and the 'when' via a concrete 'Use when deciding whether to train a custom model, estimating total cost, or calculating break-even vs frontier APIs' clause. This mirrors the score-5 example's structure; the 'when' is explicit and trigger-phrased, not merely implied as at 4.

5 / 5

Trigger Term Quality

Natural trigger phrases like 'train a custom model', 'estimating total cost', 'calculating break-even', and 'frontier APIs' are present, but common synonyms users would actually say are missing — most notably 'fine-tune'/'fine-tuning', 'fine-tuning cost', and 'custom model vs API'. This matches the anchor 'Good keyword coverage; a few natural terms missing' rather than the comprehensive synonym coverage of a 5.

4 / 5

Distinctiveness Conflict Risk

The niche (economic decision analysis for model specialization) is distinct and the triggers are cost/ROI-framed rather than generic. However, it would plausibly co-trigger with adjacent skills the body itself references (e.g. 'tinker-training-cost', or a general fine-tuning how-to skill) when a user asks about training costs — 'minor overlap risk with closely related skills' per the anchor-4 wording, not the minimal-conflict profile of a 5.

4 / 5

Total

18

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository

synthetic-sciences/openscience

Path

backend/cli/skills/ml-training/model-economics/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.