Name: testland/flake-pattern-reference
Rating: 98.2 (1 reviews)
Author: testland

testland/flake-pattern-reference

Reference catalog of the eight flake patterns - async/timing, test ordering, shared parallel state, resource leaks, network, locator drift, environment variance, randomness - with detection heuristics, remediation per pattern, and the concrete code-level fixes: replacing fixed sleeps with framework auto-waits, isolating state in beforeEach fixtures, per-worker DB schemas via workerIndex, try/finally teardown, mocking network + clock at the boundary, stable role-based locators, TZ pinning, and RNG seeding. Use when triaging an unknown flake to identify the category before bisecting, or when a classified flake needs the specific code change to apply.

1.07x

Quality

91%

Does it follow best practices?

Impact

99%

1.07x

Average score across 10 eval scenarios

Securityby

Passed

No findings from the security scan

Quality

Content

86%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable reference catalog with concrete APIs, a triage decision tree, and a validated worked example, supported by real one-level reference files. The only weakness is incidental framing prose that slightly inflates token cost.

Suggestions

Trim the ISTQB terminology note and the Google-correlation paragraph in the intro; the catalog's value is the patterns and fixes, not the provenance commentary.

Consider condensing the 'A flake is rarely random...' opening paragraph since the triage decision tree already conveys the same point.

Move any per-pattern detail that duplicates the reference files out of the body so the SKILL.md stays a lean overview.

Dimension	Reasoning	Score
Conciseness	The body is mostly dense, actionable signal tables and remediation bullets, but includes framing prose (the ISTQB terminology note, the Google correlation paragraph, and the 'A flake is rarely random' intro) that could be trimmed without losing utility — efficient with minor over-explanation, so 4 not 5.	4 / 5
Actionability	Provides fully executable, copy-paste-ready guidance: named APIs (await expect(loc).toBeVisible(), faker.seed(N), testInfo.workerIndex), concrete commands (jest --randomize, pytest --random-order), and a complete before/after TypeScript example covering the common case.	5 / 5
Workflow Clarity	Although it is explicitly a reference catalog rather than a workflow, the triage decision tree plus the worked example (Classify → Apply fix → Re-measure at N=300) give a clear sequence with a validation checkpoint; it stops at 4 because the primary workflow is delegated to sibling skills rather than fully self-contained with explicit feedback loops.	4 / 5
Progressive Disclosure	Clear overview catalog in the body with three well-signaled, one-level-deep reference files (timing-and-isolation-fixes.md, network-and-locator-fixes.md, environment-and-randomness-fixes.md) that exist in ./references/, plus cross-links to sibling skills; content is appropriately split and easy to navigate.	5 / 5
	Total	18 / 20 Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, specific description that clearly states both what the skill provides and when to invoke it, with concrete code-level fix actions rather than vague claims. Minor gap: it lacks a few common synonyms a user might say ('flaky test', 'intermittent failure').

Dimension	Reasoning	Score
Specificity	Lists multiple concrete code-level fixes (auto-waits, beforeEach isolation, per-worker schemas via workerIndex, try/finally teardown, boundary mocking, role-based locators, TZ pinning, RNG seeding) — comprehensive coverage of specific actions, matching the score-5 anchor.	5 / 5
Completeness	Explicitly answers both 'what' (reference catalog of eight patterns with heuristics, remediation, and code fixes) and 'when' ('Use when triaging an unknown flake... or when a classified flake needs the specific code change to apply') with concrete trigger phrases.	5 / 5
Trigger Term Quality	Natural terms 'flake', 'triaging an unknown flake', 'bisecting', and 'classified flake' map to what practitioners say, but common synonyms like 'flaky test' or 'intermittent test failure' are absent — good coverage with a few natural terms missing, so 4 not 5.	4 / 5
Distinctiveness Conflict Risk	Occupies a clear niche (flake-pattern triage/classification) with distinct triggers and minimal overlap risk with other skills; it is not generic.	5 / 5
	Total	19 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

17 days ago

Table of Contents

Discovery Implementation Validation