CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/flake-pattern-reference

Reference catalog of the eight flake patterns - async/timing, test ordering, shared parallel state, resource leaks, network, locator drift, environment variance, randomness - with detection heuristics, remediation per pattern, and the concrete code-level fixes: replacing fixed sleeps with framework auto-waits, isolating state in beforeEach fixtures, per-worker DB schemas via workerIndex, try/finally teardown, mocking network + clock at the boundary, stable role-based locators, TZ pinning, and RNG seeding. Use when triaging an unknown flake to identify the category before bisecting, or when a classified flake needs the specific code change to apply.

98

1.07x
Quality

91%

Does it follow best practices?

Impact

99%

1.07x

Average score across 10 eval scenarios

SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-8/

{
  "context": "Every test checks out a connection from a five-connection pool and none of them ever gives one back, so the file works until it has more tests than the pool has connections and then fails at the sixth slot regardless of which test occupies it. Predicted baseline failure: the agent takes one of the two options already on the table - raise `max`, or split the file - both of which buy slots rather than return connections, and both of which fail again at the new limit. A subtler predicted miss is releasing on the last line of each test body: it passes today and leaks again the first time an assertion fails partway through, which is precisely when a test run is already in trouble. Diagnoses are predicted to describe 'the pool running out' without naming the missing return of a finite resource, and to skip the reason the failure tracked the position in the file rather than the test.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Deliverables present and green",
      "description": "`node --test` passes with all seven originally-named tests present and executing, and `pool-notes.md` exists. A test deleted, skipped, or moved to another file scores zero here.",
      "max_score": 6
    },
    {
      "name": "Every acquired connection is returned",
      "description": "Each test returns its connection to the pool - typically acquiring in a per-test setup hook and releasing in a per-test teardown hook, or wrapping the body so the release runs on the way out. Full credit requires no acquire path left without a matching release. A release placed on the final line of each test body scores at most half: a failing assertion above it skips the release and the exhaustion returns exactly when the suite is already red. Reaching green with connections still leaked scores zero.",
      "max_score": 28
    },
    {
      "name": "Release survives a failing test",
      "description": "The return of the connection is guaranteed whether the test passed or threw - a teardown hook the runner always executes, or `try` / `finally` around the body. A fix whose release only runs on the success path scores zero for this criterion.",
      "max_score": 14
    },
    {
      "name": "The file proves it leaves nothing checked out",
      "description": "The test file itself asserts that the suite ends with `pool.stats().inUse === 0`. Missing this check scores zero here; asserting it only inside one test rather than after the suite scores at most half.",
      "max_score": 10
    },
    {
      "name": "Notes name the unreturned resource and the position effect",
      "description": "`pool-notes.md` states that each test consumed one of five connections and never released it, that the failure therefore lands on the sixth checkout regardless of which test that is - which is why it followed the slot and not the test - and that a test passing on its own proves nothing about a suite that shares a finite pool. A write-up that says only 'the pool is exhausted' without naming the missing release scores at most half. Blaming `src/pool.js`, the max, or the runner scores zero.",
      "max_score": 20
    },
    {
      "name": "No raised limit, retry, or file split as the fix",
      "description": "MUST NOT reach green by raising `max` above 5, creating a second pool, creating a pool per test to sidestep the cap, retrying the acquire, splitting the file so fewer tests share the pool, or skipping tests. Any of these offered as the fix scores zero for this criterion. The notes must also say why a larger `max` only moves the failure to a later slot.",
      "max_score": 14
    },
    {
      "name": "Source unchanged",
      "description": "MUST NOT modify anything under `src/` - not the pool's default max, not `release` to tolerate double-returns, not the repository. Any edit there scores zero here.",
      "max_score": 8
    }
  ]
}

SKILL.md

tile.json