CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

kiln-model-sweep

The scheduled model sweep. Reads the team's models channel for hints, discovers new LLM models, adds them to ml_model_list.py, runs the paid tests, marks dead provider entries deprecated, opens PRs under the team's rules, announces them in Slack, and checks whether remote_config needs a publish PR. Use when asked to run the model sweep by hand, to add or deprecate models unattended, or to change how the daily sweep behaves.

73

Quality

92%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

SKILL.md
Quality
Evals
Security

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An exceptionally actionable, well-sequenced operational prompt: exact commands, formats, and validation gates with feedback loops for every risky operation, delegating detail to one-level-deep external skills. The minor weaknesses are trimmable dated asides and rationale digressions, and a long gates section that could be split into reference files.

Suggestions

Move the dated team-decision asides (e.g., 'Team decision of 2026-10-20', 'Team decision of 2026-10-01') into a short 'Decision log' section or the referenced specs directory so the operational rules stay timeless.

Split the long 'Pre-answered gates' list into topical subsections (or a reference file) so individual gates like thinking levels are easier to navigate.

Tighten rationale digressions such as the squash-merge explanation on the remote-config rules to one sentence plus a pointer to the specs, keeping the body lean.

DimensionReasoningScore

Conciseness

The body is dense and purposeful — commands, gates, and formats with almost no padding — but contains minor trimmable material: dated asides ('Team decision of 2026-10-20', 'Team decision of 2026-10-01') and long rationale digressions such as the squash-merge explanation and the very long thinking-levels gate. It fits 'efficient; minor instances of over-explanation that could be trimmed' rather than 'every token earns its place'.

4 / 5

Actionability

Fully executable throughout: exact commands ('uv run ruff check', 'KILN_TEST_COMPATIBILITY=1 uv run pytest -q libs/core/kiln_ai/adapters/test_remote_config.py::test_backwards_compatibility_with_v0_19', 'git merge-tree --write-tree origin/remote_config origin/main'), exact MCP tool names, exact branch patterns ('model-sweep/adds-YYYY-MM-DD'), and copy-paste-ready Slack message and PR-header templates covering the common cases.

5 / 5

Workflow Clarity

Clear sequencing (skills 1→2→3, gates, then 'Remote config publish check, every run, after the PRs above') with explicit validation checkpoints and error-recovery loops: targeted checks must pass before commit, 'Retry a failing test once with -n 0', the compatibility test and conflict check before a publish PR, and 'first confirm that git diff <sha> origin/remote_config is empty' with an else-branch, plus an end-of-run report checklist. Guardrails ('Never merge, approve, or force-push') cover the batch/destructive-adjacent operations.

5 / 5

Progressive Disclosure

No bundle files exist, so this scores the body's structure: it delegates cleanly to three clearly-signaled, one-level-deep skill paths ('.agents/skills/claude-maintain-models/SKILL.md', etc.) and a specs directory, with good section headers. It falls short of a 5 because the long mixed-topic 'Pre-answered gates' list and the inlined PR-body/Slack templates could live in separate reference files, fitting 'good structure; minor organization gaps'.

4 / 5

Total

18

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: concrete, comprehensive action list with an explicit 'Use when' clause containing natural trigger phrases, all in third person. The only weakness is slightly thin synonym coverage in its trigger terms.

DimensionReasoningScore

Specificity

The description lists eight concrete actions ('Reads the team's models channel for hints, discovers new LLM models, adds them to ml_model_list.py, runs the paid tests, marks dead provider entries deprecated, opens PRs... announces them in Slack, and checks whether remote_config needs a publish PR'), naming a specific file and branch — comprehensive coverage matching the top anchor.

5 / 5

Completeness

It explicitly answers both 'what' (the full action list) and 'when' with concrete trigger phrases ('Use when asked to run the model sweep by hand, to add or deprecate models unattended, or to change how the daily sweep behaves'), matching the top anchor; a 4 would require a less explicit 'when'.

5 / 5

Trigger Term Quality

'run the model sweep by hand', 'add or deprecate models unattended', and 'change how the daily sweep behaves' are natural user phrases with a synonym ('daily sweep'), but coverage lacks further variations, so it fits 'good keyword coverage; a few natural terms missing' rather than the comprehensive-synonyms anchor at 5.

4 / 5

Distinctiveness Conflict Risk

It names a clear internal niche (ml_model_list.py, remote_config, the models channel) with distinct triggers, so it is unlikely to fire for a wrong skill; there is no meaningful overlap with generic skills.

5 / 5

Total

19

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

Kiln-AI/Kiln

Path

.agents/skills/kiln-model-sweep/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.