claude-maintain-models

Add new AI models to Kiln's ml_model_list.py and produce a Discord announcement. Use when the user wants to add, integrate, or register a new LLM model (e.g. Claude, GPT, DeepSeek, Gemini, Kimi, Qwen, Grok) into the Kiln model list, mentions adding a model to ml_model_list.py, asks to discover/find new models that are available but not yet in Kiln, or wants to add a net-new AI provider to Kiln.

Quality

89%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Quality

Content

85%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An exceptionally actionable, well-sequenced workflow with strong validation gates and domain-specific gotchas that justify most of its length. Its main weakness is progressive disclosure: the sizeable reference sections (provider quirks, thinking levels, slug lookup, lagging providers) are inlined in SKILL.md instead of being split into one-level-deep reference files.

Suggestions

Move the Provider Quirks Reference, Thinking Levels Reference, and Slug Lookup Reference sections into separate files under references/ (e.g. provider_quirks.md, thinking_levels.md, slug_lookup.md), leaving concise summaries and links in SKILL.md so the main file stays a navigable overview.

Tighten the Reasoning Capable Default and Net-New Provider sections to bullet-point essentials; some explanatory prose reiterates motivation that could be trimmed without losing the actionable rule.

Consider extracting the PR body template and test-output format rules into a short reference file, keeping only the gate rules and checklist inline in Phase 5.

Dimension	Reasoning	Score
Conciseness	Mostly efficient and heavily focused on Kiln-specific knowledge Claude would not know (ordering rules, test-env gotchas, provider quirks), with only minor instances of over-explanation that could be trimmed; it is long but the length is largely earned.	4 / 5
Actionability	Fully executable guidance throughout: exact curl+jq catalog queries, precise pytest invocations with -k bracket syntax, git branch/commit commands, and a copy-paste PR body template covering the common cases.	5 / 5
Workflow Clarity	Five clearly sequenced phases with explicit validation checkpoints (4b smoke test, 4e confirm failures are pre-existing, 5.1 gate before pushing) and feedback loops (fix one test -> re-run -> verify -> full suite); a final checklist reinforces the sequence.	5 / 5
Progressive Disclosure	Well-structured with clearly signaled same-file reference sections (Provider Quirks, Thinking Levels, Slug Lookup, Lagging Providers), but this is a ~790-line monolithic SKILL.md with no bundle files — the large reference material that clearly belongs in separate reference files is inlined rather than split out.	3 / 5
	Total	17 / 20 Passed

Description

95%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, third-person description that crisply states what the skill does and gives rich, concrete trigger conditions covering adding, discovering, and net-new-provider scenarios. Only minor: it enumerates just two concrete actions, leaving the internal code-editing steps to the body.

Dimension	Reasoning	Score
Specificity	Names two concrete, specific actions ("Add new AI models to Kiln's ml_model_list.py" and "produce a Discord announcement") with a concrete file target, sitting above anchor 3 because the actions are precise rather than generic, but below 5 since only two actions are enumerated.	4 / 5
Completeness	Explicitly answers both what (add models to ml_model_list.py + Discord announcement) and when via a clear "Use when the user wants to..." clause with concrete trigger phrases for multiple scenarios.	5 / 5
Trigger Term Quality	Comprehensive natural-term coverage with synonyms ("add, integrate, or register"), named model families (Claude, GPT, DeepSeek, Gemini, Kimi, Qwen, Grok), the concrete file ml_model_list.py, and trigger verbs ("discover/find", "net-new AI provider").	5 / 5
Distinctiveness Conflict Risk	Occupies a clear niche tied to Kiln's specific ml_model_list.py with distinct triggers (provider list, model registration); minimal risk of triggering for unrelated skills.	5 / 5
	Total	19 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
skill_md_line_count	SKILL.md is long (790 lines); consider splitting into references/ and linking	Warning

	Total	15 / 16 Passed

Repository: Kiln-AI/Kiln
Path: .agents/skills/claude-maintain-models/SKILL.md
Commit: ff39535

Reviewed: ~22 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.