Agent release gate

Product-level sanity QA for the agent runtime, one layer below the playground UI. The question is not "is every detail right" — it is "if a user opens the product and does the obvious first things, do they work?" This is the gate a release passes before shipping.

Every check asserts on the wire (the SSE frame types the browser sees) and on side effects (the file really persisted, the revision really incremented) — never on what the model says. That makes it deployment-agnostic: point it at any stack and the assertions still hold.

Run it

Set three environment variables for the deployment under test, then run the gate:

export AGENTA_BASE=https://your-stack.example.com   # deployment origin
export AGENTA_PROJECT_ID=...                         # target project
export AGENTA_API_KEY=...                            # project API key

uv run resources/qa_product.py --all --custom-slug <vault-slug>  # every cell, every journey
uv run resources/qa_product.py --cell P1                         # one cell
uv run resources/qa_product.py --cell C1 --only chat              # one journey

Paths are relative to this skill's directory. The deployment's vault must hold the provider keys the cells use (Anthropic / OpenAI / OpenRouter). If the three env vars are unset the driver stops immediately and names exactly what is missing; a legacy --env-file <path> fallback also exists. --all includes cell P2 (a custom OpenAI-compatible provider), which needs a vault slug passed via --custom-slug; the driver fails fast if it's missing. Cell S1 (the Codex subscription path) additionally needs the subscription sidecar logged in on the target deployment — see resources/coverage.md for what each cell requires.

Reading the result. Each journey prints PASS, FAIL, or SKIP with a one-line reason, and a per-cell markdown table lands with the full JSON in ./qa-gate-runs/<timestamp>/ (override the location with AGENTA_QA_RUNS_DIR). Runs are written to the current working directory, never into the skill. SKIP is expected where a journey does not apply to a cell (for example mcp on any Pi cell — user MCP is Claude-only). Any FAIL blocks the release until triaged.

When results lie

The runtime fails open: a component can break, get logged, and the turn still succeeds with a normal-looking answer. A green turn is therefore not proof on its own. Before trusting a pass, read resources/LESSONS.md — every trap there produced a green test that proved nothing. The two that bite hardest: replay conversation history byte-faithfully (tool parts included) or every turn silently goes cold, and re-run any prior blocker-level finding after a redeploy before believing it.

Resources (read on demand)

• resources/coverage.md — the cells (harness × sandbox × auth) and journeys (chat, mount, tool, approve, deny, commit, warm, mcp) with a one-line meaning for each.
• resources/LESSONS.md — the traps. Read before writing or trusting any agent QA test.
• resources/qa_product.py — the gate driver (cells × journeys).
• resources/qa_probe.py — a one-turn wire probe: uv run resources/qa_probe.py confirms the product path answers at all before running the full gate.
• resources/qa_longctx.py — optional long-context / Gmail / concurrent-session probes. Needs live Gmail and GitHub Composio connections in the target project; skip it otherwise.
• resources/seeds/ — representative green results.json files kept as regression-seed references.

Release-night findings and the full evidence history are archived in docs/design/agent-workflows/projects/qa/ (STATUS.md, findings.md, matrix.md).