heuristic-test-design-reference

Overview

The exploratory-testing literature converged on four canonical heuristic test-design models, each cited inline at point of use below. This skill is a pure reference - no execution steps; it is the catalog test-case-from-live-feature and exploratory-charter authoring consume to generate coverage when no documented input exists.

When to use

• A feature is being tested with no story, no AC, no documentation (the dominant real-world starting point for exploratory testing).
• A legacy / brownfield codebase has no test coverage and you are onboarding cold.
• A competitor or reverse-engineered product is under review (security audit, market intel).
• A spec exists but is so thin that heuristic supplementation is needed alongside it.
• A team's exploratory charter needs a structured prompt set instead of pure intuition.

Do not use this skill alone to produce a deliverable. It is the input to a downstream authoring skill (test-case-from-live-feature, manual-test-script-author). The catalog tells you what to look at; the downstream skill turns observations into a matrix or charter.

How to use

Run the four models in sequence; each one narrows the next. No written story or acceptance criteria are required - that is the whole point of the catalog.

1. Enumerate coverage targets - walk SFDPOT (Model 1) across the feature to list every structure, function, data element, platform, operation, and time dimension worth probing.
2. Attack each target - apply Whittaker's attack patterns (Model 2) to each SFDPOT item to turn "what to cover" into concrete "how it can break".
3. Classify surprises - when a behaviour looks wrong and no spec says so, name the FEW HICCUPPS consistency lens (Model 3) it violates, so the finding is a defensible bug report.
4. Cross-check quality dimensions - pass the feature through the ISO/IEC 25010 characteristics (Model 4) to catch performance, security, usability, or reliability gaps a functional walk misses.
5. Hand off - feed the SFDPOT plus Whittaker walk to test-case-from-live-feature (matrix) or manual-test-script-author (script). This skill produces prompts, not the deliverable.

The catalog entries below define each model; the "combine the models" table and the worked example show the sequence applied end to end.

Model 1 - HTSM / SFDPOT product elements (Bach)

James Bach's Heuristic Test Strategy Model (HTSM v6.3) is the canonical "guideword heuristics" framework. The mnemonic SFDPOT covers the Product Elements dimension - the parts of the system that need coverage. The four HTSM focus areas are: Test Techniques, Project Elements, Product Factors, and Quality Criteria categories.

Each guideword expands the search space. SFDPOT applied to "checkout flow" generates: Structure (cart service, payment service, inventory service), Function (add to cart, apply coupon, choose shipping, pay, confirm), Data (cart items, coupon codes, addresses, payment tokens, order IDs), Platform (desktop / mobile, iOS / Android, Stripe / Adyen integrations), Operations (deploy, rollback, monitoring, alerting), Time (cart expiry, coupon expiry, payment timeout, idempotency keys).

Coverage check: a feature passed through SFDPOT that has zero notes under one guideword is a flag - either the guideword is genuinely n/a (rare) or the team has a coverage gap.

Model 2 - Whittaker "How to Break Software" attack patterns

James Whittaker's How to Break Software (cited in the exploratory-testing literature as the canonical attack-pattern catalog) organises adversarial test ideas as attacks - explicit ways the software can fail. The canonical attack categories:

Apply Whittaker after SFDPOT: SFDPOT enumerates what to cover; Whittaker enumerates how each thing can break.

Model 3 - FEW HICCUPPS consistency oracles (Bolton)

Michael Bolton's FEW HICCUPPS is the canonical oracle heuristic - how do you decide a behavior is wrong when no spec says so? Each letter is a consistency lens:

A finding that violates at least one consistency lens is a defensible bug report even without a spec. The lens is the oracle.

Model 4 - ISO/IEC 25010 quality characteristics

The canonical quality-attribute taxonomy from ISO/IEC 25010 (the system / software product quality model, successor to ISO 9126). The eight characteristics define what kinds of quality a feature can have - beyond "does it work":

The 2023 revision adds Safety and Interaction Capability as additional top-level characteristics (cite by stable ID - ISO/IEC 25010:2023; the canonical ISO page sits behind a Cloudflare challenge). Apply 25010 alongside SFDPOT: SFDPOT enumerates what to cover; 25010 enumerates which kinds of quality to test for. A feature can be functionally correct but fail on performance, security, or usability - and 25010 is the prompt that reminds the tester to check.

How to combine the models

The four models are orthogonal:

The models are applied in the order given in "How to use" above: SFDPOT enumerates targets, Whittaker attacks each one, FEW HICCUPPS classifies the surprises, and 25010 confirms no quality dimension was skipped.

Worked example

A full end-to-end walk of the four models against a zero-spec brief ("test the new checkout flow, no spec"), producing SFDPOT targets, Whittaker attacks, ISO 25010 cross-checks, and FEW HICCUPPS oracle verdicts, is in references/worked-example.md.

Anti-patterns

Limitations

• Coverage breadth, not depth. The four models surface what to look at; they don't tell you how deep to go on each. Risk-based prioritisation (per risk-matrix) is the depth selector.
• Domain knowledge is still required. Applying SFDPOT to "checkout" without knowing what checkout is produces shallow output. The models are scaffolding for domain reasoning, not a replacement for it.
• Citation-grade only where canonical. HTSM is Bach; FEW HICCUPPS is Bolton; ISO 25010 is the standard; Whittaker is the book. Other heuristic frameworks exist (Crispin/Gregory's testing quadrants, Heusser's "test ideas") and are also valid - this skill names the four most-cited; the team can extend.
• No automation. This skill produces prompts; the downstream skills (test-case-from-live-feature, manual-test-script-author) turn prompts into deliverables.
• Not a substitute for product / requirements work. Heuristic test design surfaces the coverage gap; the documentation gap is a separate problem the team should escalate.

Hand-off targets

• Turn the heuristic walk into a test-case matrix → test-case-from-live-feature.
• Turn the heuristic walk into a manual execution script → manual-test-script-author (in the qa-manual-testing plugin).
• When a written spec exists, prefer the spec-driven path → test-case-ideation-from-story.
• Risk-based prioritisation of which guidewords matter most → risk-matrix.
• Domain-specific test-data generation for the attack patterns → negative-test-generator, malicious-payload-bank, boundary-value-generator.

References

Each model's primary source is cited inline at its section: HTSM / SFDPOT (satisfice.com), FEW HICCUPPS (developsense.com), ISO/IEC 25010 (Wikipedia), and Whittaker's attack patterns (via the exploratory-testing article). Additional references, not repeated inline:

• James Bach blog - heuristics category (SFDPOT, CRUSSPIC STMPL, consistency heuristics): https://www.satisfice.com/blog/archives/category/heuristics
• ISTQB glossary - exploratory testing: https://glossary.istqb.org/en_US/term/exploratory-testing
• ISTQB glossary - heuristic evaluation: https://glossary.istqb.org/en_US/term/heuristic-evaluation
• test-case-from-live-feature - downstream skill that consumes this catalog.