CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

ci-failure-triage

Decides what kind of failure a red CI test is before anyone starts fixing it. Extracts seven failure signals from the runner output, stack trace, run history, and environment metadata, then walks an ordered first-match-wins rule set to exactly one verdict: flaky-known, environment-drift, defect, timeout, flaky-pre-incident, or flake-of-unknown-cause. Emits the verdict together with the alternatives that were rejected and the specific condition each one failed, so the triage decision is auditable rather than asserted. Use when a test has just gone red and the next action depends on whether the cause is a product defect, a non-deterministic test, or drifted infrastructure.

1.12x
Quality

94%

Does it follow best practices?

Impact

99%

1.12x

Average score across 10 eval scenarios

SecuritybySnyk

Low

Low-risk findings worth noting

The canonical home for this skill is testland/ci-failure-triage

SKILL.md
Quality
Evals
Security

Quality

Content

92%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A high-quality instruction-only skill body: a deterministic, executable classification procedure with concrete match patterns, ordered rules, a fixed output template, and real, well-signaled bundle references. The only room for improvement is trimming a few explanatory paragraphs that add length without adding method.

DimensionReasoningScore

Conciseness

Efficient and dense — signal table, pattern table, ordered rules, output template — with only minor over-explanation that could be trimmed (the Google 84% motivation paragraph, the fault/defect/error terminology Claude largely knows, and the ISTQB-404 aside), placing it just below the every-token-earns-its-place anchor.

4 / 5

Actionability

Fully concrete guidance throughout: exact match strings (SIGSEGV, ENOENT, strict mode violation), conjunctive first-match-wins rule conditions, a fixed copy-paste verdict template, and explicit conventions to state (windows used, N threshold). As an instruction-only skill its guidance is fully actionable without code.

5 / 5

Workflow Clarity

Clear three-step sequence (extract signals → walk rules in order → emit verdict) with explicit checkpoints: all seven signals recorded before any rule, confidence gated on signal availability, the missing-vs-negative-signal distinction, and the mandatory Not-classified-as audit block. The operation is read-only classification, so the destructive/batch validation cap does not apply.

5 / 5

Progressive Disclosure

Well-signaled one-level-deep references to real files — the worked example and anti-pattern catalogue each get a link plus a one-line description of their contents — while the core method stays inline. Content is appropriately split and easy to navigate.

5 / 5

Total

19

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: concrete, third-person, comprehensive on what it does, with an explicit and specific 'Use when' clause. The only weakness is modest synonym coverage in trigger terms — natural phrasings like 'intermittent failure' or 'failing test' are missing alongside the present 'flaky'/'non-deterministic' terms.

DimensionReasoningScore

Specificity

Lists multiple concrete actions covering the full capability pipeline — "Extracts seven failure signals from the runner output, stack trace, run history, and environment metadata", "walks an ordered first-match-wins rule set to exactly one verdict", "Emits the verdict together with the alternatives that were rejected" — in third person, with no coverage gaps, matching the comprehensive-coverage anchor.

5 / 5

Completeness

Explicitly answers both questions: the what (classify a red CI test into one of six named verdicts with rejected alternatives recorded) and the when ("Use when a test has just gone red and the next action depends on whether the cause is a product defect, a non-deterministic test, or drifted infrastructure") with concrete trigger phrases.

5 / 5

Trigger Term Quality

Good natural-term coverage — "red CI test", "a test has just gone red", "flaky", "non-deterministic test", "product defect", "drifted infrastructure", "stack trace" — but common user phrasings like "intermittent failure", "failing test", or "is this a flake" are absent, so it falls short of the comprehensive-synonyms anchor at 5.

4 / 5

Distinctiveness Conflict Risk

Clear niche — pre-fix failure classification — with distinct triggers, and it explicitly delimits itself from adjacent work ("before anyone starts fixing it", classification not report-writing or remediation), minimizing conflict risk with general debugging or bug-report skills.

5 / 5

Total

19

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

testland/qa

Path

plugins/qa-bug-repro/skills/ci-failure-triage/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.