CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/confirmation-testing-workflow

Procedure for proving that a claimed defect fix actually reached the build under test and actually works. Covers the merge-base ancestry check that proves the running build contains the fix commit rather than trusting a version label, the priority order for choosing which reproduction to re-run, and the VERIFIED / NOT FIXED / BLOCKED verdict table whose governing rule is that any ambiguous, flaky, or unreproducible result resolves to BLOCKED and is never guessed. Scoped to ISTQB confirmation testing (does this specific fix work?), not regression testing (did the fix break something else?), and not triage or severity assignment. Use when a developer has marked a defect Fixed and someone must decide whether it moves to Verified or back to Reopened.

99

1.03x
Quality

94%

Does it follow best practices?

Impact

100%

1.03x

Average score across 10 eval scenarios

SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

task.mdevals/scenario-9/

BUG-6017 - two testers could not make the overwrite happen again on staging

Problem Description

BUG-6017: when two people edit the same document, the second save silently overwrites the first. The fix is 2ea5f19, merged last week; staging is on 74b0c62 and the containment check is in ops/checks.txt.

Two testers paired on staging yesterday and could not make the overwrite happen - the second save produced a "this document changed elsewhere" banner and left the first edit in place. Their notes are in ops/staging-run-2026-08-13.txt. The developer's unit test is green too.

The same release also shipped a client-side draft cache that puts up a similar banner, which is why I want this written down properly rather than agreed in the standup. Product wants the ticket closed before the customer call at 11:00.

Output Specification

  1. Write qa-record/BUG-6017.md: whether this defect can move to Verified, and the evidence, in a form the next person can re-check exactly. If it cannot move, state the specific reason the observations available do not settle it, and the concrete action that would.
  2. Do not edit config/features.json, src/documents.js, the existing test, or anything under ops/. Do not change any environment's configuration.

Input Files

Extract the following files before beginning.

=============== FILE: issues/BUG-6017.md ===============

BUG-6017 - Concurrent edits silently overwrite each other

Status: Fixed (awaiting verification) Reported: 2026-06-30 by k.mbeki Fix commit: 2ea5f19 on main, merged 2026-08-07

Reproduction steps

  1. User A and user B both open document doc_1 at version 3.
  2. User A saves. Version becomes 4.
  3. User B saves, still sending baseVersion 3.
  4. Observed: B's save succeeds and A's text is gone, with no warning. Expected: B's save is rejected as stale and B is told to reload.

Comments

2026-08-07 r.haddad: Fixed in 2ea5f19 - saves now carry the base version and a stale one is rejected. Behind strictDocumentVersioning while we watch the rejection rate.

=============== FILE: package.json =============== { "name": "docs-service", "version": "5.6.0", "private": true, "scripts": { "test": "node --test" } }

=============== FILE: config/features.json =============== { "default": { "strictDocumentVersioning": false }, "environments": { "production": { "strictDocumentVersioning": true }, "staging": { "strictDocumentVersioning": false }, "dev": { "strictDocumentVersioning": true } } }

=============== FILE: src/documents.js =============== 'use strict';

const features = require('../config/features.json');

function flagsFor(env) { return { ...features.default, ...(features.environments[env] || {}) }; }

function saveDocument(store, env, { id, body, baseVersion }) { const doc = store[id]; if (!doc) { return { status: 404, code: 'NOT_FOUND' }; } if (flagsFor(env).strictDocumentVersioning && baseVersion !== doc.version) { return { status: 409, code: 'STALE_VERSION', currentVersion: doc.version }; } doc.body = body; doc.version += 1; return { status: 200, version: doc.version }; }

module.exports = { saveDocument, flagsFor };

=============== FILE: tests/documents.test.js =============== 'use strict';

const test = require('node:test'); const assert = require('node:assert/strict'); const { saveDocument } = require('../src/documents');

function store() { return { doc_1: { id: 'doc_1', body: 'original', version: 3 } }; }

test('a stale save is rejected', () => { const s = store(); assert.deepEqual(saveDocument(s, 'dev', { id: 'doc_1', body: 'A', baseVersion: 3 }), { status: 200, version: 4, }); assert.deepEqual(saveDocument(s, 'dev', { id: 'doc_1', body: 'B', baseVersion: 3 }), { status: 409, code: 'STALE_VERSION', currentVersion: 4, }); assert.equal(s.doc_1.body, 'A'); });

=============== FILE: ops/checks.txt =============== $ curl -s https://docs.staging.internal/internal/build-info {"service":"docs-service","commit":"74b0c62","branch":"main","deployedAt":"2026-08-12T16:10:27Z"}

$ git merge-base --is-ancestor 2ea5f19 74b0c62; echo $? 0

=============== FILE: ops/staging-run-2026-08-13.txt =============== Paired run, staging, 2026-08-13 14:05-14:20 UTC. n.farah + l.pereira.

Doc doc_1 opened in two browsers, both showing version 3. n.farah saved "quarterly numbers updated". saved, no error l.pereira saved "added the appendix" ~20s later. banner: "This document changed elsewhere - reload to continue" Reopened doc_1: n.farah's text is there, l.pereira's is not.

Could not reproduce the overwrite in 3 further attempts. Calling it fixed.

SKILL.md

tile.json