offline-fallback-tests

Overview

Offline behavior fragments into eight named recipes per Jake Archibald's offline cookbook off-cookbook - and most PWAs use three or four of them across different routes. A test suite that asserts "the site works offline" misses the per-route recipe contract: a route on Cache only must serve from cache even when network is fine; a route on Network falling back to cache must prefer network when both are present.

This skill emits the per-route test suite. The output is a Playwright spec file plus a coverage YAML that maps each URL pattern to its recipe and its assertion. The suite is distinct from the strategy authoring in sw-cache-strategy-author (in the qa-modern-web plugin); that skill authors the strategy, this builder generates the verification.

Composes with:

• workbox-tests - when the SW uses Workbox's offlineFallback() recipe, the workbox-tests spec covers the recipe's runtime; this builder covers the per-route decision and the page-side assertion.
• service-worker-lifecycle-tests - every recipe assumes an active SW; the lifecycle spec is the prerequisite for this one.
• pwa-install-flow-reference - Stage 1's service-worker prerequisite cell is the same SW this builder tests offline behavior of.

When to use

• New PWA - author the offline-fallback test plan before any route ships SW caching.
• "Site doesn't work offline" bug report - the recipe matrix localizes whether the recipe is wrong or just missing for that route.
• Migrating from hand-rolled caches.match / event.respondWith to Workbox offlineFallback() - assert that the recipe-named behavior matches the hand-rolled one.
• Adding a new offline-capable feature (e.g. saved articles, draft form data) - pick a recipe per the matrix and emit the test.

Workflow

Step 1 - Inventory routes and pick a recipe per route

For each URL pattern the SW intercepts, decide which of the eight named recipes per off-cookbook applies. The cookbook recipes verbatim:

Workbox layers a named composite recipe on top per wb-recipes:

• offlineFallback() - "serve a web page, image, or font if there's a routing error" when users are offline, defaulting to offline.html.

Capture decisions in a YAML manifest:

# tests/offline-recipe-matrix.yaml
routes:
  - pattern: /_next/static/**
    recipe: Cache only
    reason: Hashed filenames, immutable
  - pattern: /
    recipe: Network falling back to cache
    reason: HTML shell - fresh while online, stale offline
  - pattern: /api/feed
    recipe: Cache then network
    reason: List view; show fast, update behind
  - pattern: /api/user/me
    recipe: Network only
    reason: PII; never cache
  - pattern: /img/**
    recipe: Cache, falling back to network
    reason: Bandwidth save; image lifetime ≥ cache TTL
  - pattern: /*
    recipe: Generic fallback
    reason: Catch-all 404; offline.html

Step 2 - Emit the per-recipe Playwright test

For each recipe row in Step 1's matrix, emit the matching test - the assertion shape must distinguish "served from cache" vs "served from network." The two poles (cache-always vs network-always) are inline; the rest live in the reference.

Cache only

Serves from cache even with network reachable AND with network blocked:

test('Cache only: /_next/static/x.css serves from cache regardless of network', async ({ page, context }) => {
  await page.goto('https://localhost:3000/');
  await page.waitForLoadState('networkidle');

  // Block network for the static path
  await context.route('**/_next/static/**', route => route.abort('failed'));

  const res = await page.evaluate(() =>
    fetch('/_next/static/x.css').then(r => r.status).catch(() => 0)
  );
  expect(res).toBe(200);
});

Network only

test('Network only: /api/user/me fails when offline (never cached)', async ({ page, context }) => {
  await page.goto('https://localhost:3000/');
  await context.setOffline(true);
  const res = await page.evaluate(() =>
    fetch('/api/user/me').then(r => r.status).catch(() => 0)
  );
  expect(res).toBe(0); // network error; not served from cache
});

The remaining six recipe templates - Cache falling back to network, Cache and network race, Network falling back to cache, Cache then network, Generic fallback, and SW-side templating - are in references/recipes.md, each with its assertion-shape note.

Step 3 - Test the underlying storage choice

Per off-data, offline data needs the right storage:

Test the Storage Manager persist() request:

test('Storage Manager persist() succeeds for installed PWA', async ({ page }) => {
  await page.goto('https://localhost:3000/');
  const result = await page.evaluate(async () => {
    const persisted = await navigator.storage.persist();
    const isPersisted = await navigator.storage.persisted();
    return { persisted, isPersisted };
  });
  expect(result.isPersisted).toBe(true);
});

Per off-data: "Chrome permits origins to use up to 60% of total disk space." Tests that assume small caches mask quota-pressure bugs; assert the application stays well below the estimate:

test('cache usage stays under 50MB', async ({ page }) => {
  await page.goto('https://localhost:3000/');
  await page.waitForLoadState('networkidle');
  const est = await page.evaluate(async () => navigator.storage.estimate());
  expect(est.usage!).toBeLessThan(50 * 1024 * 1024);
});

Step 4 - Test the offline-fallback page itself

Per wb-recipes, offlineFallback() defaults to offline.html. Assert the file is precached + the response shape:

test('offline.html is precached and well-formed', async ({ page, context }) => {
  await page.goto('https://localhost:3000/');
  await page.waitForLoadState('networkidle');

  let [sw] = context.serviceWorkers();
  if (!sw) sw = await context.waitForEvent('serviceworker');

  const offlinePagePresent = await sw.evaluate(async () => {
    for (const name of await caches.keys()) {
      const c = await caches.open(name);
      const found = await c.match('/offline.html') || await c.match('/_offline');
      if (found) return found.status;
    }
    return null;
  });

  expect(offlinePagePresent).toBe(200);
});

If the project overrides the default pageFallback option, swap the assertion target.

Step 5 - Test the offline → online recovery

A subtle class of bugs: a route correctly serves from cache offline, but doesn't refresh when network returns. Test the transition:

test('cache then network: recovers fresh data when online returns', async ({ page, context }) => {
  await page.goto('https://localhost:3000/feed');
  await context.setOffline(true);
  await page.reload();
  // Stale cache served

  await context.setOffline(false);
  // Trigger refresh
  await page.evaluate(() => fetch('/api/feed').then(r => r.json()));
  await page.waitForTimeout(500);

  // Assert no "offline" banner lingering
  await expect(page.locator('[data-testid="offline-banner"]')).not.toBeVisible();
});

Step 6 - Emit the coverage matrix and CI gate

Write tests/offline-coverage.yaml mapping each route from Step 1 to its test name and the source cookbook recipe:

matrix:
  - route: /_next/static/**
    recipe: Cache only
    spec: offline-fallback.spec.ts > 'Cache only: /_next/static/*'
    source: off-cookbook "Cache only"
  - route: /
    recipe: Network falling back to cache
    spec: offline-fallback.spec.ts > 'Network falling back to cache'
    source: off-cookbook "Network falling back to cache"
  - route: /api/feed
    recipe: Cache then network
    spec: offline-fallback.spec.ts > 'Cache then network: /api/feed'
    source: off-cookbook "Cache then network"
  - route: /api/user/me
    recipe: Network only
    spec: offline-fallback.spec.ts > 'Network only: /api/user/me'
    source: off-cookbook "Network only"
  - route: /img/**
    recipe: Cache, falling back to network
    spec: offline-fallback.spec.ts > 'Cache, falling back to network: /img/*'
    source: off-cookbook "Cache, falling back to network"
  - route: /*
    recipe: Generic fallback
    spec: offline-fallback.spec.ts > 'Generic fallback'
    source: off-cookbook "Generic fallback"; wb-recipes offlineFallback()
storage:
  - check: cache-usage-under-budget
    source: off-data Storage Manager estimate()
  - check: persisted-storage-granted
    source: off-data Storage Manager persist()

CI gates on every row having a passing spec.

Worked example: a 4-route news PWA

Applying Step 1's matrix to a news PWA - static -> Cache only, / -> Network falling back to cache, /api/articles -> Cache then network, /api/user/me -> Network only, /* -> Generic fallback - emits a 5-cell Playwright spec that runs in ~15 seconds on Chromium and catches the four most common offline regressions:

1. Static cache miss on stale deploy.
2. HTML shell stuck stale when network returns.
3. Article list never refreshes when going online.
4. PII endpoint accidentally cached.

Anti-patterns

Limitations

• The "Cache and network race" recipe is timing-sensitive; Step 2's < 100ms threshold is a heuristic for "cache won" - slow test machines or instrumented Playwright may report higher values. Tighten / loosen per the project's CI host.
• SW-side templating is rare in modern PWAs; the Step 2 test for it assumes the project actually uses the pattern. Skip if not.
• Quota-pressure tests are fragile - a 50 MB ceiling per the Step 3 test is project-specific; per off-data the browser permits "up to 60% of total disk space" which varies by device.
• navigator.storage.persist() may prompt the user on some browsers; in headless CI the prompt auto-dismisses (or auto-grants) inconsistently. Use the permissions: ['persistent-storage'] Playwright context option to grant upfront.
• offline.html precache depends on the SW's precacheAndRoute(self.__WB_MANIFEST) including the offline page; workbox-tests Step 3 asserts the manifest contents, which is the upstream gate.
• No IndexedDB-specific recipe in off-cookbook; this builder documents IDB usage in the storage matrix (Step 3) but does not generate per-store test cells. Pair with a dedicated IDB test skill for full coverage.

References

• web.dev - Offline cookbook (the eight named recipes; verbatim use-case statements per recipe) - off-cookbook.
• web.dev - Learn PWA: Offline data (Cache Storage vs IndexedDB vs Storage Manager; quota note "up to 60% of total disk space") - off-data.
• Chrome - Workbox recipes (offlineFallback() recipe defaults to offline.html) - wb-recipes.
• Composes: workbox-tests, service-worker-lifecycle-tests, pwa-install-flow-reference.
• Differentiation: sw-cache-strategy-author authors the strategy; this builder generates the verification suite. Both consume the cookbook recipe vocabulary; the authoring side writes the SW code, this side writes the spec that locks it.
• Sibling builders: add-to-homescreen-flow-tests, service-worker-lifecycle-tests.