Playwright E2E testing patterns — web-first assertions, user-visible locators, network interception, fixtures, authentication, and parallel execution
98
99%
Does it follow best practices?
Impact
98%
1.81xAverage score across 5 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent proactively applies Playwright best practices when writing E2E tests for a search/filter page with API interactions. The task does not mention locator strategy, waiting patterns, API mocking, or test structure.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Web-first assertions",
"description": "All element checks use expect(locator).toBeVisible(), .toHaveText(), .toHaveCount(), .toContainText(). No page.$(), page.textContent(), or page.evaluate().",
"max_score": 14
},
{
"name": "User-visible locators",
"description": "Elements selected with getByRole, getByLabel, getByText, getByPlaceholder, or getByTestId -- not CSS selectors.",
"max_score": 14
},
{
"name": "getByPlaceholder or getByRole for search",
"description": "The search input is located using getByPlaceholder('Search products...') or getByRole('searchbox'/'textbox') -- not a CSS selector like page.locator('input.search').",
"max_score": 8
},
{
"name": "No explicit waits",
"description": "No page.waitForTimeout() or page.waitForSelector() before actions. Uses web-first assertions to wait for filtered results to appear.",
"max_score": 12
},
{
"name": "waitForResponse for API calls",
"description": "The agent uses page.waitForResponse() (set up BEFORE the triggering action like search input or filter change) to wait for the API to return results, rather than arbitrary timeouts.",
"max_score": 10
},
{
"name": "page.route for API mocking",
"description": "At least one test uses page.route() to mock /api/products with controlled data for deterministic assertions (e.g., verifying exact product count or testing empty results).",
"max_score": 10
},
{
"name": "test.describe grouping",
"description": "Related tests are grouped in test.describe blocks (e.g., one for search, one for filtering, one for sorting).",
"max_score": 8
},
{
"name": "test.beforeEach for navigation",
"description": "Common setup (page.goto('/products')) is in test.beforeEach, not repeated in every test.",
"max_score": 6
},
{
"name": "webServer configured",
"description": "playwright.config.ts includes webServer with command, port, and reuseExistingServer.",
"max_score": 8
},
{
"name": "Screenshot and trace config",
"description": "Config includes screenshot: 'only-on-failure' and trace: 'on-first-retry'.",
"max_score": 6
},
{
"name": "Glob patterns for route matching",
"description": "page.route() and page.waitForResponse() use glob patterns (e.g., '**/api/products*') rather than exact full URLs.",
"max_score": 4
}
]
}