Use for any question about which test automation framework or QA tool to pick, including quick ones such as Playwright or Cypress for a new app, or a stack for web, API and mobile: Playwright vs Cypress vs Selenium, Appium vs Espresso/XCUITest, RestAssured vs Karate, k6 vs Locust. Compares tradeoffs, matches project needs to a choice, covers commercial vendor evaluation and records the decision.
95%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
Low
Low-risk findings worth noting
Decision support for picking a test stack (framework, runner, reporter, CI shape): score the project's needs, compare per layer, then record one decision or an explicit deferral. It does not generate boilerplate; after the choice, the per-framework skill (playwright-testing, cypress-testing, ...) configures it.
Do not use this skill when:
playwright-testing, cypress-testing, etc.).test-pyramid-balancer.qa-mobile plugin's documentation directly.Six NFR axes drive framework choice. Score each 1 - 5 for the project; rank them by priority. The framework matrix in Step 2 uses these scores.
| NFR axis | Question |
|---|---|
| Cross-browser scope | Is multi-browser execution required? (Chromium-only? + Firefox + WebKit? + IE/Edge legacy?) |
| Mobile scope | Real device + emulator? Hybrid app webview? Native-only? Or web-mobile-viewport only? |
| Team language | What languages do the engineers already know? (Avoiding the framework-language mismatch is the #1 maintenance cost.) |
| Execution speed | Parallel-shard target - minutes for the full suite. CI-cost-driven? |
| Ecosystem maturity | Third-party integrations the team needs (visual regression, accessibility, perf, contract). |
| Hire-ability | Can the team hire engineers familiar with the framework? Smaller frameworks → smaller talent pool. |
Checkpoint before Step 2. Write the six scores down, each with the evidence behind it (a manifest line, a CI wall-clock figure, the team's language list, a roadmap item with a date), and get the team to confirm the ranking. If a top-ranked axis cannot be scored from evidence today (language not decided, no stable product surface), stop and go to Step 6 (defer) instead of reading the matrix. A roadmap line ("Android in H2") is a condition that would reopen the decision, not a score.
| Framework | Cross-browser | Mobile | Language | Speed (parallel) | Ecosystem | Hire-ability | Notes |
|---|---|---|---|---|---|---|---|
| Playwright | Chromium / Firefox / WebKit native, all in one runtime | Mobile-viewport emulation; experimental Android support (Chrome for Android, WebView) (Android API) | TS / JS / Python / .NET / Java | Excellent (auto-parallel, sharding built-in) | Strong (trace viewer, visual snapshots, fixtures, MCP integration) | High | A common default for greenfield web E2E; its built-in auto-waiting removes the manual-wait flakiness that dominates Selenium suites (Playwright actionability). |
| Cypress | Chromium-family + Firefox; WebKit only behind experimentalWebKitSupport (experiments) | Mobile viewport only; no real-device | JS / TS only | Good (parallel via Cypress Cloud; CLI-parallel limited) | Strong (huge plugin ecosystem) | High | Strong DX for component testing; runs inside-browser limits cross-origin and iframe scenarios. It cannot control more than one open browser at a time, and a flow that opens a new tab (3-D Secure, OAuth pop-ups) needs the @cypress/puppeteer plugin (trade-offs). |
| Selenium / WebdriverIO | All browsers via WebDriver protocol | Real device via Appium | All major languages (Java / Python / C# / JS / Ruby) | Moderate (Selenium Grid; WDIO improves on Selenium's runner) | Mature (oldest ecosystem) | Highest (historical talent pool) | Mature but more flake-prone than Playwright: async-wait is the single largest flake category at 45% (Luo et al. 2014), and Selenium leaves that synchronization manual. Migration target, not greenfield default. |
| TestCafe | All browsers; proxy-based (no WebDriver) | Mobile via emulators | JS / TS | Moderate | Smaller ecosystem | Lower | Niche; integrated runner. |
| Puppeteer | Chrome and Firefox (stable Firefox from v23, supported browsers) | Limited | JS / TS | Good | Smaller than Playwright | Lower | Browser automation library rather than a test runner; pair it with Jest or similar. |
Decision tree for greenfield web E2E:
The per-layer matrices (mobile native, API / contract, performance) are in references/other-layers.md. The short rules:
pact-contract-testing).After the team has chosen a stack, adopt the canonical directory layout the per-framework skill assumes - a convention, not a mandate, but the starting point a newcomer can read. The layouts for Playwright + Jest, Cypress + Mocha, and Selenium / WebdriverIO are in references/directory-layouts.md.
Universal across frameworks:
| Concern | Convention |
|---|---|
| Parallelisation | Shard by file (Playwright --shard=X/Y, Cypress Cloud, WDIO maxInstances). Aim for 5 - 10 minute wall-clock for the full suite per shard. |
| Retries | Retry once on first failure; never retry locally (only CI). Tests retried >1× are flake candidates - triage them. |
| Trace / video | Capture on-first-retry (off for green runs to save storage). Playwright trace: 'on-first-retry' is the default; Cypress + cypress-video-trim similar. |
| Reporting | JUnit XML output for the CI's test-result panel; Allure for human reporting; both via plugin. |
| Secrets | Load from CI secret store (GitHub Actions Secrets, GitLab CI Variables); never commit. |
| Environment matrix | One job per (framework, browser, environment) cell; do not mix in one job. |
The skill recommends deferring framework choice when:
In these cases, the right output is an explicit deferral note: "no decision today; revisit when (a)/(b)/(c) resolves."
When the decision is a commercial tool - a test-management platform (TestRail / Qase / Xray / Zephyr), a no-code automation vendor (mabl / Testim / Functionize), a visual-regression service (Applitools / Percy / Chromatic), or an AI copilot tier - the axes change: contract, lock-in, and exit cost matter as much as capability. Run the seven-axis vendor evaluation (capability fit, cost model, integration depth, lock-in risk, exit cost, contractual posture, customer-reference data) in references/vendor-evaluation.md. It emits an evidence pack with a weighted score per the team's NFR order and deliberately refuses to pick the winner - the team owns the procurement choice.
Whichever path produced the choice, write it down as a decision record so a later reader can see the signal, the rejected alternative, and what would reverse it. The ADR-based format - observed project signal, exactly one primary recommendation, a why-not clause, one read-next link, and mandatory flip conditions - is in references/decision-record-format.md.
| Anti-pattern | Why it fails | Fix |
|---|---|---|
| Picking the framework before the NFRs are scored | Choice driven by hype, not fit; high migration cost when the wrong framework can't deliver. | Step 1 - score the NFRs first. |
| Standardising on one framework across every test layer | Different layers need different tools (Playwright for E2E ≠ k6 for perf ≠ Pact for contract). | Pick per layer; the stack is multiple frameworks. |
| Picking Selenium for greenfield in 2026 | Manual async-wait is the dominant flake category at 45% (Luo et al. 2014), and Selenium does not auto-wait. | Use Playwright for greenfield; reserve Selenium for legacy maintenance. |
| Cross-language teams picking a single-language framework | Engineers can't contribute; suite becomes one person's domain. | Either pick a multi-language framework (Playwright / Selenium) or commit to retraining. |
| Adopting a framework because a contractor used it | Contractor leaves; team can't maintain. | Hire-ability is an NFR. |
| Skipping the directory-layout convention | Every newcomer authoring tests in a different shape; review burden grows. | Step 4 - pick a canonical layout up front, even if you deviate later. |
| Treating this skill as "framework recommender" rather than "decision support" | The skill recommends; the team decides. Automating the decision strips accountability. | The output of this skill is a documented choice, not an automatic install. |
playwright-testing, cypress-testing, selenium-testing, webdriverio-testing).test-pyramid-balancer.experimentalWebKitSupport): https://docs.cypress.io/app/references/experiments@cypress/puppeteer): https://docs.cypress.io/app/references/trade-offsplaywright-testing, cypress-testing, selenium-testing, webdriverio-testing - downstream per-framework configuration skills.test-pyramid-balancer - layer-mix tuning after the framework is in place.