CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

flake-axis-bisection

Use when one test is known to fail intermittently and reading its code has not explained why. Varies one condition at a time (isolation, order, workers, viewport, latency, repetition), counts passes and failures with confidence intervals, and walks shared resources once parallelism is implicated.

Quality

87%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

The canonical home for this skill is testland/flake-axis-bisection

SKILL.md
Quality
Evals
Security

Quality

Content

77%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is an unusually rigorous, well-sequenced experimental protocol with strong statistical decision rules, good bundle structure, and a concrete report contract. It is held back by padding (cited definitions, etymology, derivations of standard statistics) and by three axes that describe variations without giving an executable way to realize them.

Suggestions

Cut the opening Fowler definition and the 'flaky test' etymology paragraph; replace with one line stating the trigger condition, and move the Google size-vs-flakiness note into the existing statistics reference or delete it.

State the binomial sizing and two-proportion comparison as rules ("choose N from the table in references/...; flag |z| > 1.96") without deriving them inline — the derivations already live in the reference file.

Add an executable knob for the three naked axes: a concrete traffic-shaping command or CDP throttling call for network, and the runner flags or config for viewport and animation suppression (even one per axis, mirroring the Playwright/Jest table).

DimensionReasoningScore

Conciseness

Lookup tables are properly deferred to references and most prose is load-bearing, but the body still spends tokens on things Claude already knows: the opening Fowler definition and the etymology of "flaky test", the inline derivation of (1-p)^N = 0.05 and the pooled two-proportion z formula, and a marginal "note on where flakes concentrate" section, plus a time-sensitive CDP version-drift caution in the main body. Mostly efficient with several sections that could be tightened fits 3 rather than 4 — the removable material is more than minor — and clearly not 2, since the majority of the content instructs rather than explains.

3 / 5

Actionability

Concrete runner flags (--workers, --repeat-each, --runInBand, --randomize, --showSeed/--seed), explicit N budgets, the 1.96 threshold, and a copy-paste report template make the protocol mostly executable. Three of the eight axes (network, viewport, animation suppression) name their variations but give no executable knob, which is more than a minor gap, so 4 rather than 5; well above 3 since nothing is pseudocode.

4 / 5

Workflow Clarity

Steps 1-6 are sequenced in dependency order with an explicit decision rule as the validation checkpoint, an early-stop rule, a confirmation feedback loop (2x screening hit -> re-measure at N=60 -> apply the z rule), a required report shape, and guidance for the negative-result case. Matches the 5 anchor's explicit validation and error-recovery loop.

5 / 5

Progressive Disclosure

Two real, one-level-deep reference files (both verified present on disk and containing what the body claims) are linked in context and catalogued in a "Deep references" section describing each file's contents; the body inlines only the distilled numbers. Clear overview with well-signaled references fits the 5 anchor.

5 / 5

Total

17

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific, third-person, and answers both what and when with an unusually well-staged trigger condition. Its only weakness is the absence of the practitioner vocabulary ("flaky", "flaky test") that users most often reach for, which costs it trigger-term coverage.

DimensionReasoningScore

Specificity

"Varies one condition at a time (isolation, order, workers, viewport, latency, repetition)", "counts passes and failures with confidence intervals", and "walks shared resources once parallelism is implicated" are multiple concrete, comprehensive actions. Fits the 5 anchor rather than 4 because the actions named cover the whole protocol, not just most of it.

5 / 5

Completeness

The what (vary one axis at a time, count outcomes with confidence intervals, walk shared resources) and an explicit, staged when ("Use when one test is known to fail intermittently and reading its code has not explained why") are both clearly present with concrete triggers. Matches the 5 anchor; 4 would require the when-clause to be less explicit than it is.

5 / 5

Trigger Term Quality

"one test is known to fail intermittently", "passes and failures", and "parallelism" are natural phrases, but the most common user phrasings — "flaky test", "flaky", "test fails sometimes" — never appear. Good coverage with a few natural terms missing fits 4; it is above 3 because the terms present are ones users would actually say, and below 5 because synonyms of the single most likely trigger term are absent.

4 / 5

Distinctiveness Conflict Risk

The trigger is a narrow niche — a single intermittently failing test that code inspection has already failed to explain — which distinguishes it from catalog-matching and commit-bisection skills. Clear niche with distinct triggers and minimal conflict risk fits the 5 anchor.

5 / 5

Total

19

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

relative_links

Relative link issues: 1 missing, 1 deeper-than-1-level

Warning

Total

15

/

16

Passed

Repository

testland/qa

Path

plugins/qa-flake-triage/skills/flake-axis-bisection/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.