Lines of questioning — Alpha user research

Point 1 — Understand user needs

Ask:

• Which concepts did you put in front of users, and which users?
• How were digitally excluded or vulnerable participants recruited? If they were not, why are they not affected by these hypotheses?
• What did a participant do that surprised the team, and what did you change afterwards?
• How many rounds were there, and what was the gap between them for?
• Which finding is still a hypothesis because only one or two people showed it?

Evidence to request: discussion guides, a recruitment screener, session notes, a short iteration history (what changed, because of which session).

Answers that indicate Met: the team can narrate a round, a change, and a second round. The screener shows they tried to reach people who struggle with the channel.

Answers that indicate Not Met: participants were colleagues, or "digitally confident customers". Red line A3 when the excluded group is material to the concept.

Point 2 — Solve a whole problem for users

Ask:

• Did the concept test include the step that happens before the user reaches you, and the step that happens after you are finished?
• What did users do when the prototype ended but their task did not?
• Which concept did users reject, and did you kill it?
• Where did an intermediary (adviser, carer, caseworker) need a different concept from the end user?

Evidence to request: task scripts that cover the whole problem, notes on concepts discarded after testing.

Answers that indicate Met: at least one concept died, and the surviving concept still includes a handoff the department does not control.

Answers that indicate Not Met: every concept was a variation of the same form. Users were not asked to finish the real task.

Beta handover

Ask what the team still needs to learn with assistive technology before a public Beta. Record the answer as a recommendation unless the Alpha concept is already being treated as the live interface, in which case flag it for the design assessor under red line A2.