qa

Use when an A/B test result is read for a ship decision, especially off a dashboard, a small sample or many metrics. Computes significance per metric with multiple-comparison correction, reads confidence intervals, checks novelty, interactions, Simpson's paradox, guardrails and stop-early rules.

Use when code instrumented with Statsig, Optimizely, Split.io (Harness FME), Amplitude Experiment or VWO needs tests. Initializes the SDK offline from a committed config fixture, pins each test to an arm and checks assignment integrity, with per-vendor references.

Use when one test is known to fail intermittently and reading its code has not explained why. Varies one condition at a time (isolation, order, workers, viewport, latency, repetition), counts passes and failures with confidence intervals, and walks shared resources once parallelism is implicated.

Use when a team needs a lasting view of test reliability from CI history (JUnit XML or JSON): a flake-rate metric, a Grafana panel or Datadog CI Visibility view, the quarantine-candidate query, trend alerts, and a weekly or monthly test-health report for Slack or a wiki.

Use when a Windows desktop app needs UI tests from C# or .NET with in-process UI Automation calls: FlaUI UIA2 and UIA3 packages, launching or attaching to the app, ConditionFactory locators, Retry waits and xUnit, NUnit or MSTest. For the WebDriver protocol use winappdriver.

Use whenever a schema is managed with Flyway: versioned, repeatable and undo migration files, migrate, info, validate, baseline and repair, the schema history table, migration ordering questions, and CI gates on schema changes, via CLI, Maven, Gradle or Docker.

Use when a product processing EU or California personal data needs GDPR or CCPA/CPRA readiness tests: data subject access, erasure, portability and objection, consent and withdrawal, data residency, 72-hour breach notice, data minimization, and CCPA rights including Global Privacy Control.

Use whenever user stories, acceptance criteria, manual test steps or a spec section need to become Gherkin .feature files. Maps criteria to scenarios, finds Scenario Outlines and Background, reuses existing steps and flags hidden preconditions. Writes Gherkin only, not step code.

Use whenever the task is to run tests in GitHub Actions: a test workflow on pull requests, matrix builds across OS and runtime versions, splitting a suite across parallel runners, service containers such as Postgres or Redis, uploading JUnit reports, and trigger filters.

Use whenever the task is to run tests in GitLab CI/CD: test stages in .gitlab-ci.yml, parallel matrix jobs, junit and coverage artifact reports, Postgres or Redis services, needs between jobs, rules for triggers and retry policy.

Use when embedded C++ (C++17 or later) on ARM needs unit tests with GoogleTest: TEST, TEST_F and parameterized tests, EXPECT vs ASSERT, fixtures, GoogleMock, cross-compiling with arm-none-eabi-g++, running on host or QEMU and XML output for CI. Pure C uses unity-test-framework-c.

Use when a pull request changes a GraphQL schema, or a Federation supergraph may have drifted. Diffs the schemas with graphql-inspector, classifies changes as breaking, dangerous or safe, applies rules such as deprecated-field removal and usage checks, and gates CI.

Use whenever data validation uses Great Expectations (GX Core) on Pandas, SQL or Spark: ExpectationSuites, ValidationDefinitions and Checkpoints, running validations on batches and reading the JSON result to gate CI.

Use when scoping an embedded hardware-in-the-loop (HIL) rig or reading an automotive, aerospace or industrial HIL report: MIL, SIL, PIL and HIL stages, NI, dSPACE, Vector and Speedgoat stacks, CAN, LIN, FlexRay and Ethernet emulation, fault injection and ISO 26262, DO-178C and IEC 61508 evidence.

Use when a product handling protected health information (PHI) needs HIPAA Security Rule readiness tests: administrative, physical and technical safeguards (access control, audit logs, integrity, transmission security), PHI in fixtures, minimum-necessary access and BAA scope.

Use whenever the same request or message could be processed twice (at-least-once queues, retries, webhook redelivery, double clicks, mobile retries) and duplicates must have no extra effect. Tests idempotency keys, delivery semantics, fingerprint dedup and the idempotency window.

Use when tests must prove a request produces the expected spans across services in a running Jaeger, Zipkin or Grafana Tempo backend. Runs the backend in CI, queries traces by service and operation, and asserts on spans, parent-child links and durations.

Use whenever the task is to run tests in Jenkins: declarative Jenkinsfile test stages, parallel stages across agents, post always/failure actions, JUnit report publishing and lockable resources for shared test infrastructure.

Use whenever Kafka producers or consumers need tests, in KafkaJS, kafka-go or Spring Kafka: a real broker via Testcontainers, offset commits, consumer-group rebalances, at-least-once vs exactly-once transactions, idempotent producers and dead-letter topics.

Use when a JVM project wants HTTP API tests written as Karate .feature files instead of Java code: the Gherkin-style DSL, match with fuzzy validators, JUnit 5 and Maven runs, and JUnit XML for CI. For Java fluent tests use restassured-testing; for Python YAML tests use tavern-testing.