test-isolation-patterns

Overview

A test that fails sometimes for non-obvious reasons is non-deterministic. Per Martin Fowler - Eradicating Non-Determinism in Tests: "A test is non-deterministic when it passes sometimes and fails sometimes, without any noticeable change in the code, tests, or environment… Once you start ignoring a regression test failure, then that test is useless and you might as well throw it away." The dominant cause is broken isolation - one test affecting another, the environment leaking, fixtures sharing state. This catalog is the canonical reference for the isolation patterns that prevent it.

This skill is a pure reference - no execution steps. It is the catalog cited when auditing fixture coupling, retry/wait policy consistency, and CI integration health. It complements test-code-conventions §6 (which is the file-level rule against global-fixture hubs) with the cross-cutting architecture patterns. It also complements flake-pattern-reference, which catalogs flake symptoms; this skill catalogs the prevention patterns.

When to use

• Designing a new framework - pick the fixture scope and isolation strategy.
• Auditing an existing framework where flake-rate is rising (broken isolation drives the concurrency and test-order-dependency flake categories, together about a third of flakes per Luo et al. 2014).
• Migrating from sequential to parallel execution - the parallel-safety patterns become load-bearing.
• Refactoring fixture inheritance chains - apply the cleanup-discipline patterns.

How to use

1. Frame each test in the four phases (Pattern 1). Be explicit about where Setup and Teardown live, because Phases 1 and 4 are where isolation breaks.
2. Pick the tightest fixture scope that holds (Pattern 2). Default to per-test; escalate to per-describe or shared only when setup is measurably expensive and the group never mutates the fixture.
3. Choose Fresh Fixture (Pattern 3), or a Persistent Fresh Fixture (transaction-rollback) when a fresh rebuild is measured too slow.
4. For DB-backed or network-backed tests, pick an external-dependency strategy from references/database-store-isolation.md and references/network-service-isolation.md.
5. Before enabling parallel execution, apply the parallel-safety patterns (Pattern 4): worker-scoped state, unique IDs, ephemeral paths, no global singletons.
6. Wire teardown through the runner's afterEach by default (Pattern 5, cleanup discipline).
7. Cross-check against the pattern-selection guide and the cross-cutting anti-patterns before sign-off.

Pattern 1 - The four-phase test pattern

Canonical source: Gerard Meszaros - xUnit Test Patterns: Refactoring Test Code (2007). Referenced in the Wikipedia entry on test fixture.

Every test has four phases:

Phases 1 and 4 together are fixture management. Patterns 2 through 5 below cover how to do them safely; isolating external stores and services is covered in the deep references.

Pattern 2 - Fixture scope

The framework's test runner offers three or four scopes; the team picks the tightest scope that meets the constraint.

The single rule that prevents most flake: never share mutable fixtures across tests. If a fixture is mutated by any test, it must be per-test scoped.

Framework-specific scope syntax (illustrative; cite the per-framework skill for tool-specific details)

• Jest / Vitest: beforeEach / beforeAll (per-describe by default within a describe block).
• Playwright Test: test.beforeEach / test.beforeAll; test.use({}) for per-test config; fixtures via test.extend().
• pytest: @pytest.fixture(scope="function" | "class" | "module" | "session").
• JUnit 5: @BeforeEach / @BeforeAll; @TestInstance(Lifecycle.PER_CLASS).
• RSpec: before(:each) / before(:all).

Anti-patterns

Pattern 3 - Fresh Fixture vs Shared Fixture trade-off

Canonical source: Martin Fowler - Eradicating Non-Determinism in Tests.

Fowler's framing: "I prefer the former [Fresh Fixture], as it's often easier - and in particular easier to find the source of a problem… [but] rebuilding the database each time can add a lot of time to test runs, so that argues for switching to a clean-up strategy."

The transaction-rollback pattern (Persistent Fresh Fixture): Begin a transaction at test start; do all the test's DB work inside it; rollback at test end. The database is materially unchanged across tests. The pattern works for any DB that supports transactions; integration-test frameworks like DatabaseCleaner (Ruby), pytest-django's db fixture, Spring's @Transactional test annotation all implement it. The five DB-level strategies are catalogued in references/database-store-isolation.md.

Anti-patterns

Pattern 4 - Parallel safety

Canonical source: Fowler - Eradicating Non-Determinism in Tests on isolation as the parallel-safety prerequisite, plus Luo et al. FSE 2014, which attributes 20% of flakes to concurrency problems (race conditions and deadlocks).

Parallel execution magnifies every isolation bug. The patterns that make parallel safe:

Anti-patterns

Pattern 5 - Cleanup discipline

Canonical source: Meszaros's xUnit Test Patterns (2007) - the Garbage-Collected Teardown vs In-line Teardown vs Implicit Teardown vs Setup Decorator patterns.

The four canonical cleanup approaches:

Rule: Implicit Teardown via the runner's afterEach hook is the default. In-line Teardown is acceptable when the cleanup is specific to one test. Tagged Cleanup is for fixtures whose lifetime is variable (held across multiple tests, then released).

Anti-patterns

Worked example - diagnosing a leaking-state test

Symptom. updates a member's role passes when its file runs alone but fails intermittently in the full suite - the classic order-dependent flake (Fowler).

The suite (before):

describe("member admin", () => {
  let user;
  beforeAll(() => { user = createUser({ role: "member" }); }); // per-describe scope

  it("promotes to admin", () => {
    promote(user);
    expect(user.role).toBe("admin"); // mutates the shared fixture
  });

  it("updates a member's role", () => {
    expect(user.role).toBe("member"); // fails: user was mutated above
  });
});

Diagnosis. Walk Pattern 2: user is a per-describe (beforeAll) fixture that the first test mutates. The second test inherits the mutated state. This is the Pattern 2 anti-pattern row "Per-describe fixture that any test in the describe mutates", and it violates the single rule that prevents most flake: never share mutable fixtures across tests.

Fix. Move the fixture to per-test (beforeEach) scope so each test gets a Fresh Fixture (Pattern 3):

describe("member admin", () => {
  let user;
  beforeEach(() => { user = createUser({ role: "member" }); }); // per-test scope

  it("promotes to admin", () => {
    promote(user);
    expect(user.role).toBe("admin");
  });

  it("updates a member's role", () => {
    expect(user.role).toBe("member"); // always fresh; order-independent
  });
});

Each test now starts from a clean object - maximally isolated and parallel-safe. If createUser were genuinely expensive, the middle path is Pattern 3's Persistent Fresh Fixture (transaction-rollback), not a shared mutable object.

Deep references

Once fixture scope is chosen, the two external-dependency concerns have their own deep catalogs:

• Database / external-store isolation - the five canonical strategies (transaction-rollback, database-per-worker, template-database, containerised, in-memory) with trade-offs and anti-patterns: references/database-store-isolation.md.
• Network / external-service isolation - stub vs contract-test vs controlled-real-call, with anti-patterns: references/network-service-isolation.md.

Cross-cutting anti-patterns

Pattern-selection guide

Hand-off targets

• Per-file fixture coupling rule → test-code-conventions §6.
• Flake symptoms / pattern catalog → flake-pattern-reference (qa-flake-triage) - symptoms; this skill is the prevention reference.
• Quarantine a chronically flaky test → flaky-test-quarantine (qa-flake-triage).
• Stub / mock external services → msw-handlers, wiremock-stubs, mountebank-imposters (qa-test-data).
• Test data construction patterns → test-data-patterns (qa-test-data, sister catalog).
• Object-model architecture patterns → object-model-patterns (sister catalog).
• Test step granularity → test-step-design-patterns (sister catalog).

References

• Martin Fowler - Eradicating Non-Determinism in Tests (the load-bearing reference for Fresh-vs-Shared-Fixture trade-off and the "non-deterministic test is useless" rule): https://martinfowler.com/articles/nonDeterminism.html
• Martin Fowler - Practical Test Pyramid (cited for the in-memory-substitution anti-pattern): https://martinfowler.com/articles/practical-test-pyramid.html
• Gerard Meszaros - xUnit Test Patterns: Refactoring Test Code (2007) - the canonical reference for the four-phase test pattern and all named fixture / teardown patterns: ISBN 978-0131495050.
• Wikipedia - Test fixture (cites Meszaros's four-phase pattern): https://en.wikipedia.org/wiki/Test_fixture
• Luo et al. (FSE 2014) - An Empirical Analysis of Flaky Tests (the original academic taxonomy of flake categories: 45% async-wait, 20% concurrency, 12% test-order-dependency, from 201 fixes across 51 projects) which this catalog's patterns prevent: https://mir.cs.illinois.edu/marinov/publications/LuoETAL14FlakyTestsAnalysis.pdf
• Google Testing Blog, "Flaky Tests at Google and How We Mitigate Them" - flake prevalence (about 16% of tests show some flakiness): https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html
• Testcontainers - https://testcontainers.com/ (the canonical containerised-DB-per-test reference)
• Gerard Meszaros - Principles of Test Automation ("Principle: Keep Tests Independent", also known as Independent Test - the canonical statement of the test-isolation principle this catalog implements): http://xunitpatterns.com/Principles%20of%20Test%20Automation.html
• ISTQB glossary - test fixture: https://glossary.istqb.org/en_US/term/test-fixture
• test-code-conventions §6, flake-pattern-reference (qa-flake-triage) - companion file-level and symptom-level references.
• object-model-patterns, test-data-patterns (qa-test-data), test-step-design-patterns - sister architecture-tier pattern catalogs.