CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

testland/flake-axis-bisection

Use when one test is known to fail intermittently and reading its code has not explained why. Varies one condition at a time (isolation, order, workers, viewport, latency, repetition), counts passes and failures with confidence intervals, and walks shared resources once parallelism is implicated.

Quality

88%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

85%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is a clearly sequenced six-step protocol with explicit statistical decision rules, per-runner commands, a stopping rule, a confirmation-run feedback loop, and a copy-paste report template, with lookup tables properly deferred to two real, one-level-deep, well-signaled references. Its weakness is token efficiency: it re-derives standard statistics inline, includes terminology trivia and a background essay, and repeats content that already lives in the statistics reference.

Suggestions

Cut the terminology paragraphs (the Fowler definition of non-determinism and "The term 'flaky test' is practitioner-emergent...") — they are background Claude already knows and do not change how the protocol is executed.

Keep only the decision rule and the one load-bearing number in Steps 2, 5a, and the rule-of-three passage, and defer the binomial/Wilson/z derivations entirely to references/reproduction-rate-statistics.md, which already contains these tables and derivations — the body currently duplicates them.

Fold "A note on where flakes concentrate" down to its single practical consequence (budget the sweep against the test's own runtime; prefer N=20 screen plus one deep confirmation) and merge it into the Step 2 cost paragraph.

DimensionReasoningScore

Conciseness

"The term 'flaky test' is practitioner-emergent, popularized by the Google Testing Blog; there is no standards-body definition behind it", the inline derivation "solving (1 - p)^N = 0.05 gives the N...", the rule-of-three explanation, and "A note on where flakes concentrate" are trivia and standard statistics Claude already knows, partly duplicating tables already in the reference file. The core protocol content is dense and load-bearing, so this fits anchor 3 (mostly efficient but some unnecessary explanation); not 2 because padding is confined to a few sections, not 4 because the over-explanation is more than minor.

3 / 5

Actionability

Concrete per-runner commands ("--repeat-each <N>", "--showSeed", "--maxWorkers=<num>", "--runInBand", "--randomize"), an exact decision rule ("compare |z| against ... 1.96"), N conventions ("screen all axes at N=20, then re-measure only the baseline and the one or two implicated variations at N=60"), and a copy-paste report template with a fully worked example. For an instruction-only skill this matches anchor 5; not 4 because commands and thresholds cover the common cases with no material gaps.

5 / 5

Workflow Clarity

Six numbered steps in execution order with an explicit stopping rule ("stop early when an axis reproduces strongly: there is no prize for completing the sweep"), a dedicated validation stage (Step 5's z-rule with thresholds and caveats), and a confirmation feedback loop ("an implicated axis is confirmed by a second, deeper measurement before anyone edits code"), plus negative-result and anti-pattern handling. Matches anchor 5: clear sequence with explicit validation and error-recovery guidance.

5 / 5

Progressive Disclosure

Two references, both real files (verified on disk), one level deep, signaled at point of use ("The full p-to-N table and its derivation are in references/reproduction-rate-statistics.md") and indexed under "Deep references" with a description of each; the reference files themselves contain no nested file references. Lookup tables and the collision-class walk are appropriately split out of the protocol body, matching anchor 5; not 4 because navigation and placement leave no organization gap.

5 / 5

Total

18

/

20

Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description explicitly answers both what and when, with a concrete 'Use when' trigger whose precondition actively disambiguates it from inspection-based flake catalogs, and it names multiple specific actions with the axis list enumerated. Its one real gap is keyword coverage: 'flaky' — the term practitioners most often use — never appears. Third-person voice is used throughout.

DimensionReasoningScore

Specificity

"Varies one condition at a time (isolation, order, workers, viewport, latency, repetition), counts passes and failures with confidence intervals, and walks shared resources once parallelism is implicated" lists three concrete actions, each with named specifics, covering the whole protocol. Matches anchor 5; not 4, because coverage spans sweep, measurement, and follow-up with no material gap.

5 / 5

Completeness

"Use when one test is known to fail intermittently and reading its code has not explained why" is an explicit, concrete when-clause including a disambiguating precondition, and the rest of the sentence states the what with specifics. Matches anchor 5; not 4, because the when is fully explicit rather than merely adequate.

5 / 5

Trigger Term Quality

"one test is known to fail intermittently" and "reading its code has not explained why" are phrases a user would naturally say, but the most common practitioner term "flaky" (and synonyms like "non-deterministic" or "fails sometimes") never appears. Anchor 4 (good coverage, a few natural terms missing); not 5 because of those missing synonyms, not 3 because several genuine natural trigger phrases are present.

4 / 5

Distinctiveness Conflict Risk

The precondition "reading its code has not explained why" and the empirical framing carve a distinct niche, but the core trigger ("one test is known to fail intermittently") is shared with any sibling flaky-test skill such as a pattern catalog. Anchor 4 (mostly distinct, minor overlap with closely related skills); not 5 because that first clause alone would match other flaky-diagnosis skills.

4 / 5

Total

18

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

relative_links

Relative link issues: 1 missing, 1 deeper-than-1-level

Warning

Total

15

/

16

Passed

Reviewed

On this page