CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/confirmation-testing-workflow

Procedure for proving that a claimed defect fix actually reached the build under test and actually works. Covers the merge-base ancestry check that proves the running build contains the fix commit rather than trusting a version label, the priority order for choosing which reproduction to re-run, and the VERIFIED / NOT FIXED / BLOCKED verdict table whose governing rule is that any ambiguous, flaky, or unreproducible result resolves to BLOCKED and is never guessed. Scoped to ISTQB confirmation testing (does this specific fix work?), not regression testing (did the fix break something else?), and not triage or severity assignment. Use when a developer has marked a defect Fixed and someone must decide whether it moves to Verified or back to Reopened.

99

1.03x
Quality

94%

Does it follow best practices?

Impact

100%

1.03x

Average score across 10 eval scenarios

SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Evaluation results

100%

BUG-2264 - the endpoint in the reproduction steps no longer exists

Criteria
Without this plugin
With this plugin

Record and reproduction delivered

100%

100%

MUST NOT present the substitute run as the reported reproduction

100%

100%

The reconstruction pins the csv format

100%

100%

Step-by-step correspondence written down

100%

100%

Asserts the reported expected result, not just a 200

100%

100%

Decision supported by real output

100%

100%

Build containment recorded

100%

100%

Suite green, nothing else touched

100%

100%

100%

6%

BUG-2870 went green in CI on the fix build - can I move it on?

Criteria
Without this plugin
With this plugin

Record delivered

100%

100%

MUST NOT accept the retried, quarantined pass as confirmation

100%

100%

Cites the specific flakiness facts, not a general caution

100%

100%

Refuses to move the defect to Verified and leaves it where it is

100%

100%

Gives the concrete action that would produce usable evidence

100%

100%

Does not substitute the unit tests for the reproduction

50%

100%

States that the build containment question is settled

100%

100%

Leaves the quarantine configuration untouched

100%

100%

100%

12%

BUG-3902 - nightly is green on the fix build, release manager wants a line for the go/no-go doc

Criteria
Without this plugin
With this plugin

Record delivered

100%

100%

The evidence is the defect's own reproduction, run on its own

64%

100%

MUST NOT present the 1,842-test nightly as proof this defect is fixed

90%

100%

States what the record does not cover

95%

100%

Decision is to move the defect to Verified

100%

100%

Reproduction's pre-fix red state recorded

100%

100%

Build identity and containment recorded

100%

100%

Suite still green and nothing weakened

100%

100%

100%

BUG-4102 is marked Fixed and the release manager wants it closed before the go/no-go call

Criteria
Without this plugin
With this plugin

Record delivered

100%

100%

Refuses to move the defect to Verified

100%

100%

Names the missing build identity and the step that supplies it

100%

100%

MUST NOT treat the release note, version label or green pipeline as proof of what is running

100%

100%

Separates the local test result from the deployed build

100%

100%

The colleague's staging run is recorded as unattributable, not as a pass

100%

100%

Identifies both endpoints of the check it could not complete

100%

100%

Verdict is stated unambiguously at the top

100%

100%

100%

4%

BUG-4471 postal codes - the developer added a guard and a test, ticket is in Fixed

Criteria
Without this plugin
With this plugin

Record delivered

100%

100%

Executes the reporter's own steps against the CSV path

100%

100%

Concludes the defect is not fixed and must go back

100%

100%

MUST NOT accept the developer's test as the confirming evidence

100%

100%

Names the untouched code path precisely

100%

100%

Observed versus expected stated concretely

100%

100%

Distinguishes a code problem from a deployment problem

60%

100%

Leaves the code alone

100%

100%

100%

5%

BUG-5150 double orders - the developer says the new test covers it

Criteria
Without this plugin
With this plugin

Record delivered

100%

100%

Reproduction proven against the pre-fix copy

83%

100%

Reproduction uses the same idempotency key twice

100%

100%

MUST NOT accept the developer's test as covering this defect

100%

100%

Suite green with real output attached

100%

100%

Decision follows the reproduction, not the ticket comment

100%

100%

Build containment recorded

100%

100%

Existing files untouched

100%

100%

100%

BUG-6017 - two testers could not make the overwrite happen again on staging

Criteria
Without this plugin
With this plugin

Record delivered

100%

100%

Identifies that the fixed code path did not run in the environment observed

100%

100%

Refuses to move the defect to Verified

100%

100%

Refuses to substitute the production flag value for evidence

100%

100%

Gives the concrete action that would settle it

100%

100%

Attributes the observed banner honestly

100%

100%

MUST NOT record a conclusion without the evidence attached

100%

100%

Does not treat the containment result as sufficient

100%

100%

100%

BUG-7781 - the negative balance is gone from the wallet screen

Criteria
Without this plugin
With this plugin

Record delivered

100%

100%

Step 4 executed and the stored balance reported

100%

100%

Concludes the defect is not fixed

100%

100%

Notices step 3 does not meet its stated expectation either

100%

100%

MUST NOT accept the developer's test as confirmation

100%

100%

Names the missing behaviour and the check that would catch it

100%

100%

Each of the four steps recorded with its own result

100%

100%

Attributes the failure to the change, not the deployment

100%

100%

Code left unmodified

100%

100%

100%

BUG-9004 - containment already checked, staging behaves correctly, ticket still open

Criteria
Without this plugin
With this plugin

Record delivered

100%

100%

Re-asks containment against the commit the environment reports

100%

100%

MUST NOT treat 'the fix is on main' as a property of the running build

100%

100%

Refuses to move the defect to Verified

100%

100%

States what the 429s do and do not show

100%

100%

Gives the concrete action that closes the gap

100%

100%

Records the identifiers needed to repeat the check

100%

100%

Does not substitute the local unit test for the environment result

100%

100%

Failed

The commit on BUG-3318 does not appear to be in the staging build

Evaluated
Agent
Claude Code
Model
Claude Sonnet 4.6