CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/flaky-test-quarantine

Builds a quarantine workflow for flaky tests - marks the test with the framework's skip/fixme/retry annotation, records the failure-rate observation and a bisect link in the annotation body, sets an auto-expiry date, and produces a CI report listing every quarantined test that has expired and needs re-evaluation. Use when a flaky test is blocking the trunk and must be removed from the gating path without losing track of it.

88

1.45x
Quality

84%

Does it follow best practices?

Impact

89%

1.45x

Average score across 10 eval scenarios

SecuritybySnyk

Low

Low-risk findings worth noting

Overview
Quality
Evals
Security
Files

Quality

Content

67%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, actionable skill body with a clear multi-step workflow, real validation gates, and a properly split-out reference. The main weaknesses are minor redundancy (duplicated code block, concept re-definition) and one dangling reference.

Suggestions

Remove the duplicated test.fixme() code block — keep it in either the worked example or Step 1, not both — and drop or compress the Overview's definition of 'flaky test' since Claude already knows it.

Tighten the Cypress/Jest sections by showing the concrete tagging/skip pattern instead of deferring to 'a project-specific tagging convention parsed by your CI'.

Resolve the 'flake-pattern-reference' entry: either link it to a real bundle file under references/ or remove it from the References list to avoid a dangling pointer.

DimensionReasoningScore

Conciseness

Mostly efficient, but the Overview defines 'flaky test' (a concept Claude already knows), the worked example and Step 1 duplicate near-identical test.fixme() code blocks, and the ISTQB terminology-note blockquote is borderline padding.

3 / 5

Actionability

Playwright code is copy-paste ready with a full annotation, the annotation field table plus parseable example is concrete, and Cypress/Jest/JVM sections give canonical primitives; the only vague bit is 'introduce a project-specific tagging convention parsed by your CI'.

4 / 5

Workflow Clarity

Clear sequence (confirm → mark → annotate → TTL → report → prune) with validation gates ('Confirm it is a flake, not a regression'; 're-run the test at depth to confirm it is green') and a renewal-cap checkpoint, though the batch report job itself has no error-recovery loop.

4 / 5

Progressive Disclosure

Keeps mark/annotate/expiry/prune in front and splits the CI report to a real, clearly-linked one-level-deep reference file; minor gap is the 'flake-pattern-reference' listed under References with no path or corresponding bundle file.

4 / 5

Total

15

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, third-person description that clearly states concrete capabilities and an explicit use-trigger. It is specific, complete, and distinct; only trigger-term synonym coverage keeps it from a perfect sweep.

DimensionReasoningScore

Specificity

Lists multiple concrete actions — 'marks the test with the framework's skip/fixme/retry annotation, records the failure-rate observation and a bisect link... sets an auto-expiry date, and produces a CI report' — with comprehensive coverage of the workflow.

5 / 5

Completeness

Explicitly answers both what (builds quarantine workflow, marks, records, sets expiry, produces report) and when ('Use when a flaky test is blocking the trunk and must be removed from the gating path').

5 / 5

Trigger Term Quality

Includes natural terms a user would say ('flaky test', 'blocking the trunk', 'gating path', 'quarantine') but misses common synonyms like 'intermittent test failure' or 'flaky CI'.

4 / 5

Distinctiveness Conflict Risk

Occupies a clear niche (flaky-test quarantine) with distinct triggers and minimal overlap risk with other skills.

5 / 5

Total

19

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

Table of Contents