CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/browser-matrix-runner

Configures a CI matrix that runs the smoke / regression suite across multiple browsers per Playwright's three-engine support (Chromium, Firefox, WebKit / Safari) plus branded variants (chrome, msedge channels). Wires GitHub Actions / GitLab CI matrix syntax, captures per-browser screenshots, and aggregates per-browser pass/fail. Use when the product targets multiple browsers and the team wants automated cross-browser testing or browser-compatibility regression - e.g. a 'works in Chrome, broken in Safari' bug that needs Chrome / Firefox / Safari coverage.

75

Quality

94%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files
name:
browser-matrix-runner
description:
Configures a CI matrix that runs the smoke / regression suite across multiple browsers per Playwright's three-engine support (Chromium, Firefox, WebKit / Safari) plus branded variants (chrome, msedge channels). Wires GitHub Actions / GitLab CI matrix syntax, captures per-browser screenshots, and aggregates per-browser pass/fail. Use when the product targets multiple browsers and the team wants automated cross-browser testing or browser-compatibility regression - e.g. a 'works in Chrome, broken in Safari' bug that needs Chrome / Firefox / Safari coverage.

browser-matrix-runner

Overview

Per pw-browsers:

Playwright supports three browser engines:

  1. Chromium - "Open source builds used by default. Playwright versions support Chromium N+1 before branded browsers release it."
  2. Firefox - "Playwright's Firefox version matches the recent Firefox Stable build."
  3. WebKit - "Derived from latest WebKit main branch sources, often before Safari incorporation."

Plus branded variants (pw-browsers):

"Playwright can operate against Google Chrome and Microsoft Edge when installed on the machine. Available channels include: chrome, chrome-beta, chrome-dev, chrome-canary, msedge, msedge-beta, msedge-dev, msedge-canary."

When to use

  • The product runs on multiple browsers and the team wants automated cross-browser coverage.
  • A bug report says "works in Chrome, broken in Safari."
  • Pre-release: scheduled cross-browser smoke.

Step 1 - Install browsers

Per pw-browsers:

# All Playwright-bundled browsers
npx playwright install

# Specific browsers
npx playwright install chromium firefox webkit

# With system dependencies (CI-friendly)
npx playwright install --with-deps

Branded:

npx playwright install msedge   # only on Windows / macOS
npx playwright install chrome

Step 2 - Configure projects

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
    { name: 'firefox',  use: { ...devices['Desktop Firefox'] } },
    { name: 'webkit',   use: { ...devices['Desktop Safari'] } },
    { name: 'edge',     use: { ...devices['Desktop Edge'], channel: 'msedge' } },
    { name: 'chrome',   use: { ...devices['Desktop Chrome'], channel: 'chrome' } },
  ],
});

Step 3 - Run

# All browsers
npx playwright test

# Specific browser
npx playwright test --project=firefox

# Multiple
npx playwright test --project=chromium --project=webkit

Step 4 - CI matrix

# .github/workflows/cross-browser.yml
jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        browser: [chromium, firefox, webkit]
        include:
          - browser: edge
            os: windows-latest    # msedge on Windows
          - browser: chrome
            os: ubuntu-latest
    runs-on: ${{ matrix.os || 'ubuntu-latest' }}
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v4
        with: { node-version: '20' }
      - run: npm ci
      - run: npx playwright install --with-deps ${{ matrix.browser }}
      - run: npx playwright test --project=${{ matrix.browser }}
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: playwright-${{ matrix.browser }}
          path: playwright-report/

fail-fast: false ensures Chromium failure doesn't cancel WebKit / Firefox.

Verify each browser leg: assert npx playwright install --with-deps exited 0 and a playwright-report/ artifact was produced before counting the browser as passed. If install fails, the cause is almost always a missing system dependency - re-run with --with-deps (wired above) and surface the missing package from the installer's error output, then re-run the leg.

Step 5 - Per-browser failure analysis

When per-browser failures appear:

## Cross-browser results - `<sha>`

| Browser   | Tests | Pass | Fail | Time   |
|-----------|------:|-----:|-----:|-------:|
| Chromium  |   42  |   42 |    0 |  120s  |
| Firefox   |   42  |   42 |    0 |  135s  |
| WebKit    |   42  |   38 |    4 |  140s  |   ← WebKit-specific issues
| Edge      |   42  |   42 |    0 |  125s  |
| Chrome    |   42  |   42 |    0 |  120s  |

### WebKit-only failures

| Test                                | Symptom                                   |
|-------------------------------------|-------------------------------------------|
| `checkout.spec.ts > apply promo`     | `100vh` calculated wrong on iOS Safari   |
| `cart.spec.ts > add item`            | `IntersectionObserver` callback timing diff |
| ...                                  |                                           |

WebKit-only failures often cluster around well-known iOS Safari quirks (viewport units, scroll behavior, font rendering).

Step 6 - Engine vs channel choice

Use the engine (Playwright-bundled) whenUse the branded channel when
Default; reproducible across team / CINeed to test specific Chrome / Edge release behavior
Faster CI (lighter download per pw-browsers)Compliance / contract requires the branded build
Pre-release behavior (Chromium N+1)Bug reports against branded versions

For most teams: bundled engines for CI; branded channels for manual / spot-check.

Anti-patterns

Anti-patternWhy it failsFix
Single-browser CICross-browser regressions invisible.Matrix (Step 4).
fail-fast: true on the matrixChromium fails; team can't see WebKit / Firefox.fail-fast: false (Step 4).
Skipping --with-deps installLinux runner missing browser dependencies; tests fail.Always --with-deps in CI (Step 1).
Per-browser conditional code in testsCouples tests to browser implementation; fragile.Test what users observe; abstract per-browser quirks in production code.
Not testing branded channels at allMisses Edge / Chrome-specific bugs.Schedule weekly branded-channel runs (Step 6).

Limitations

  • WebKit on CI is approximate. Bundled WebKit ≠ Safari; per-browser quirks (especially iOS) need real-device testing via mobile-device-matrix-toolkit (in the qa-mobile plugin).
  • Edge / Chrome require host OS. msedge needs Windows or macOS; not Linux.
  • Per-browser CI cost adds up. 5 browsers × N tests = 5x CI time / cost. Use selectively for full regression; run smoke cross-browser per-PR.

References

  • pwb - Playwright supported browsers (Chromium / Firefox / WebKit + branded channels), install commands, disk-space estimates.
  • os-matrix-runner - sibling for OS / runtime matrices.
  • compatibility-budget - conventions for choosing the matrix.
  • mobile-web-emulation-runner (in the qa-mobile plugin) - sibling: mobile viewport variants of the same browsers.
Workspace
testland
Visibility
Public
Created
Last updated
Repository
https://github.com/testland/qa
Publish Source
GitHub
Badge
testland/browser-matrix-runner badge