CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

kiln-prerelease-check

Run the Kiln pre-release smoke test suite plus the standard CI checks (checks.sh), diagnose every prerelease test that broke (and why), and write a clean readable report with recommended actions. Read-only — it never edits code. Use when the user wants to validate a release candidate, run prerelease tests, or asks for a "prerelease check / smoke / audit".

72

Quality

90%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An exceptionally actionable, well-sequenced operational skill: every phase has executable commands and exact paths, diagnosis is driven by an explicit decision tree with a disciplined retry/disclosure policy, and the workflow is validated by checkpoints and a checklist. The main weaknesses are moderate repetition of the read-only and mandatory-sweep reminders, and bulk inline content (report template, sweep tables) that could live in separate reference files.

Suggestions

Consolidate the read-only reminder: it is stated ~6 times (intro, Global rules, Phase 4d, the report template, Phase 6, and the Checklist). One prominent statement in Global rules plus the existing checklist item would preserve the safety guarantee while trimming tokens (conciseness).

Move the ~77-line report template (Phase 5) and the coverage/probe sweep tables into a references/ file (e.g. references/report-template.md) linked from Phase 5, keeping SKILL.md as the workflow overview (progressive_disclosure).

The newer-sibling examples in Phase 4b (e.g. 'gpt_4o_mini while gpt_5_x_mini exists', 'claude_sonnet_4_5 while claude_sonnet_4_6/4_7 exists') are time-sensitive version references that will go stale — replace them with a family-generic rule, since the ml_model_list.py ordering rule of thumb already covers the judgment call (conciseness).

DimensionReasoningScore

Conciseness

Nearly every token is project-specific operational data Claude cannot know (marker wiring, whitelist paths, env vars, retry policy, report format). The read-only constraint is restated ~6 times ('This skill is read-only... ever', 'Read-only. Make no edits', 'the skill never made any edits', checklist item) and the 'mandatory sweep' rule ~4 times — minor instances of over-explanation that could be consolidated. Not 5 (repetition exists); not 3 (there is no generic concept explanation, and the padding is scattered one-liners reinforcing a safety constraint).

4 / 5

Actionability

Fully executable throughout: exact commands for every phase ('uv run ./checks.sh --agent-mode', the pytest invocations with -o "addopts="), a copy-paste Python heredoc for whitelist status checks, concrete grep pipelines with justification comments, exact file paths and env-var names, and a complete report template. Placeholders like '<failing::node::id>' are appropriate parameterization of copy-paste commands, not pseudocode.

5 / 5

Workflow Clarity

Six clearly sequenced phases with explicit validation checkpoints (capture checks.sh exit code; non-zero → keep going but mark run failed; skip vs. fail distinction), a 5-class diagnosis decision tree, an explicit feedback loop (retry likely-transient failures once, reclassify as transient vs. reproducible, always disclose retries — 'Never silently swap a first-run FAIL for a PASS'), and a closing checklist. This matches the 5 anchor: clear sequence, explicit validation, error-recovery loops, and a checklist.

5 / 5

Progressive Disclosure

No bundle files exist; the body is a well-sectioned single file (Global rules, Phases 1-6, Checklist) and the one external reference (specs/projects/code_tools/cross_os_spawn_checklist.md) is clearly signaled one level deep. However, the ~77-line inline report template and the long coverage/probe tables are bulk content that could be split into reference files, which keeps it below the 5 anchor. Not 3: structure is strong and references are clearly signaled, not buried.

4 / 5

Total

18

/

20

Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it names the concrete actions (run smoke suite + CI checks, diagnose failures, write a report), states the read-only boundary, and includes an explicit 'Use when...' clause with natural trigger phrases and synonyms. The only weakness is that a couple of the skill's real capabilities (notably the model-pin staleness sweep) are not surfaced in the description.

DimensionReasoningScore

Specificity

Lists several concrete actions — 'run the Kiln pre-release smoke test suite plus the standard CI checks (checks.sh)', 'diagnose every prerelease test that broke (and why)', 'write a clean readable report with recommended actions' — with only a minor coverage gap: the mandatory model-pin staleness sweep, a core capability of the skill, is not mentioned. Not 3 (well beyond 1-2 actions); not 5 (a real capability is omitted).

4 / 5

Completeness

Clearly answers both: an explicit multi-action 'what' and an explicit trigger clause — 'Use when the user wants to validate a release candidate, run prerelease tests, or asks for a "prerelease check / smoke / audit"' — with concrete trigger phrases, matching the 5 anchor exactly. Not 4: the 'when' clause is fully explicit and specific, not weaker.

5 / 5

Trigger Term Quality

Good natural-phrase coverage including synonyms: 'validate a release candidate', 'run prerelease tests', 'prerelease check / smoke / audit'. A few natural variants (e.g. 'smoke test', 'release check') are missing, so it falls short of the comprehensive 5 anchor.

4 / 5

Distinctiveness Conflict Risk

'Kiln pre-release' anchors a clear niche with distinct, domain-specific triggers (prerelease check / smoke / audit); minimal conflict risk with other skills. Not 4: there is no meaningful overlap with related skills — the domain anchoring is unambiguous.

5 / 5

Total

18

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

Kiln-AI/Kiln

Path

.agents/skills/kiln-prerelease-check/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.