CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/ci-failure-triage

Decides what kind of failure a red CI test is before anyone starts fixing it. Extracts seven failure signals from the runner output, stack trace, run history, and environment metadata, then walks an ordered first-match-wins rule set to exactly one verdict: flaky-known, environment-drift, defect, timeout, flaky-pre-incident, or flake-of-unknown-cause. Emits the verdict together with the alternatives that were rejected and the specific condition each one failed, so the triage decision is auditable rather than asserted. Use when a test has just gone red and the next action depends on whether the cause is a product defect, a non-deterministic test, or drifted infrastructure.

75

Quality

94%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

85%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable triage method with a clear sequenced workflow and clean progressive disclosure. The only weakness is conciseness: motivational and definitional prose pad an otherwise lean skill.

Suggestions

Trim or cut the opening "cost of getting it wrong" paragraph and the error/defect/failure terminology definition — these explain concepts Claude already knows and add tokens without changing the method.

Compress the ISTQB "no entry for flaky test" API digression to a single line ("ISTQB has no flaky-test entry; cite Fowler instead") to remove the procedural lookup narrative.

Move the "Why this order" rationale into the anti-patterns reference or fold one sentence into each rule, keeping the rule statements themselves primary.

DimensionReasoningScore

Conciseness

The body is mostly dense and actionable, but several passages explain concepts Claude already knows — the opening "cost of getting it wrong" stats paragraph, the error/defect/failure terminology definition ("A red test is a failure: an error introduces a defect..."), and the ISTQB-API "no entry for flaky test" digression. It is not a 3 because these could be tightened or cut without losing the method; it is not a 1 because the core signal/rule/verdict material is lean and earns its tokens.

2 / 3

Actionability

Concrete executable guidance throughout: a seven-row signal table with exact "what to record" values, a failure-mode pattern table with specific matching tokens (SIGSEGV, NoSuchElementException, strict mode violation), six rules with explicit boolean conditions over S1–S7, and a fixed verdict template with named fields. It is not a 2 because nothing is left as vague direction or pseudocode.

3 / 3

Workflow Clarity

A clear Step 1 → Step 2 → Step 3 sequence with explicit checkpoints: "Record all seven before applying any rule", confidence-tracks-signal-availability rules, and a mandatory Not-classified-as audit block. It is not a 2 because validation/verification gates are explicit rather than implicit.

3 / 3

Progressive Disclosure

SKILL.md is the overview with two clearly signaled, one-level-deep references — [references/worked-example.md] and [references/anti-patterns.md] — both of which are real files, with content appropriately split out of the main body. It is not a 2 because navigation is explicit and references are shallow rather than nested.

3 / 3

Total

11

/

12

Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, third-person description that states concrete actions, an explicit use-when trigger, and a distinctive niche. Every dimension lands at the top of the scale with no over-claiming or fluff.

DimensionReasoningScore

Specificity

Lists multiple concrete actions — "Extracts seven failure signals from the runner output, stack trace, run history, and environment metadata", "walks an ordered first-match-wins rule set to exactly one verdict", and "Emits the verdict together with the alternatives that were rejected" — matching the score-3 anchor for several specific concrete actions. It is not a 2 because the actions are enumerated and specific rather than naming only a domain plus partial actions.

3 / 3

Completeness

Explicitly answers what (decides the failure class via seven signals and an ordered rule set) and when ("Use when a test has just gone red and the next action depends on whether the cause is a product defect, a non-deterministic test, or drifted infrastructure"). It is not a 2 because the trigger clause is explicit rather than implied.

3 / 3

Trigger Term Quality

Natural trigger terms a CI user would say appear directly: "a test has just gone red", "product defect", "non-deterministic test", and "drifted infrastructure". It is not a 2 because coverage spans the common phrasings users reach for, not a single keyword with missing variations.

3 / 3

Distinctiveness Conflict Risk

Occupies a narrow, well-defined niche (CI failure triage to one of six named verdicts) with triggers unlikely to fire for unrelated skills. It is not a 2 because the scope is too specific to plausibly overlap with general testing or debugging skills.

3 / 3

Total

12

/

12

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

Table of Contents