CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/confirmation-testing-workflow

Procedure for proving that a claimed defect fix actually reached the build under test and actually works. Covers the merge-base ancestry check that proves the running build contains the fix commit rather than trusting a version label, the priority order for choosing which reproduction to re-run, and the VERIFIED / NOT FIXED / BLOCKED verdict table whose governing rule is that any ambiguous, flaky, or unreproducible result resolves to BLOCKED and is never guessed. Scoped to ISTQB confirmation testing (does this specific fix work?), not regression testing (did the fix break something else?), and not triage or severity assignment. Use when a developer has marked a defect Fixed and someone must decide whether it moves to Verified or back to Reopened.

99

1.03x
Quality

94%

Does it follow best practices?

Impact

100%

1.03x

Average score across 10 eval scenarios

SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

task.mdevals/scenario-7/

BUG-4102 is marked Fixed and the release manager wants it closed before the go/no-go call

Problem Description

BUG-4102 (archived users still receive the weekly digest) was marked Fixed on 2026-08-06. The release notes for 2.14.0 list it, staging was redeployed on 2026-08-11 with a green pipeline, and a colleague walked the reporter's steps on staging the next morning and no digest arrived.

It is the last ticket still open against the release. The release manager has asked me twice to move it to Verified before the 16:00 go/no-go call, and team policy is that a defect only moves to Verified after it has been confirmed on staging - not on a laptop.

ops/staging-build-info.json is the literal response staging's /internal/build-info endpoint returns; those are all the fields it has.

Output Specification

  1. Write qa-record/BUG-4102.md: your decision on whether this defect can move to Verified, plus the evidence behind it in a form a reviewer could re-check next week without asking you anything. If it cannot move yet, the file must say precisely what is missing and the concrete step that would supply it.
  2. Do not edit issues/BUG-4102.md, src/digest.js, tests/digest.test.js, or anything under ops/. Do not change what is deployed to staging.

Input Files

Extract the following files before beginning.

=============== FILE: issues/BUG-4102.md ===============

BUG-4102 - Archived users still receive the weekly digest

Status: Fixed (awaiting verification) Reported: 2026-07-29 by s.okafor Component: digest-worker Fix commit: 4b91ce7 on main - "digest: skip archived recipients", merged 2026-08-06

Reproduction steps

  1. Create a user with a verified email and digest_opt_in = true.
  2. Archive the user: POST /admin/users/:id/archive.
  3. Trigger the weekly digest run for that workspace.
  4. Observed: the archived user receives the digest. Expected: archived users are skipped.

Reproduced 4 times out of 4 on staging before the fix.

Comments

2026-08-06 e.reyes: Fixed in 4b91ce7. Added a unit test.

=============== FILE: package.json =============== { "name": "digest-worker", "version": "2.14.0", "private": true, "scripts": { "test": "node --test" } }

=============== FILE: src/digest.js =============== 'use strict';

function selectRecipients(users) { return users.filter((u) => u.digestOptIn && u.emailVerified && !u.archived); }

function buildDigest(users, items) { return selectRecipients(users).map((u) => ({ to: u.email, itemCount: items.length, })); }

module.exports = { selectRecipients, buildDigest };

=============== FILE: tests/digest.test.js =============== 'use strict';

const test = require('node:test'); const assert = require('node:assert/strict'); const { selectRecipients } = require('../src/digest');

const USERS = [ { email: 'a@example.com', digestOptIn: true, emailVerified: true, archived: false }, { email: 'b@example.com', digestOptIn: true, emailVerified: true, archived: true }, ];

test('archived recipients are skipped', () => { assert.deepEqual( selectRecipients(USERS).map((u) => u.email), ['a@example.com'], ); });

=============== FILE: ops/staging-build-info.json =============== { "service": "digest-worker", "version": "2.14.0", "releaseName": "aug-week-2", "deployedAt": "2026-08-11T14:02:11Z", "deployPipeline": "green", "region": "eu-central-1" }

=============== FILE: ops/release-2.14.0.md ===============

Release 2.14.0 - aug-week-2

Deployed to staging 2026-08-11 14:02 UTC. Deploy pipeline green.

Included:

  • BUG-4102 archived users receive the weekly digest
  • BUG-4390 CSV export header row duplicated
  • FEAT-221 workspace-level digest schedule

Cut from release/2.14, branched 2026-08-07.

=============== FILE: ops/staging-run-2026-08-12.txt =============== Ran by m.duarte, 2026-08-12 09:40 UTC, staging admin API + Mailhog.

  1. Created qa-4102@example.com, digest_opt_in on, email verified. OK
  2. POST /admin/users/u_88213/archive -> 204 OK
  3. Triggered digest for workspace w_qa OK
  4. Mailhog inbox for qa-4102@example.com after 5 minutes: empty.

No digest arrived. Looks fixed to me.

SKILL.md

tile.json