Lines of questioning — Live user research

Point 1 — Understand user needs

Ask:

• What research has happened since go-live, on what cadence, and with whom?
• How do live users tell you something is wrong, other than by abandoning the service?
• When did you last sit with someone using assisted digital, or listen to contact-centre calls?
• Show a change shipped because of research in the last two quarters.
• Which user group has appeared since launch that the Beta sample missed?

Evidence to request: research plan for the live service, recent session or visit notes, a feedback backlog, the change it produced.

Answers that indicate Met: a cadence that has actually happened, including people who struggle, and a line from a finding to a release.

Answers that indicate Not Met: research ended at launch, or the only voice is a complaints mailbox nobody analyses. Red line L5.

Point 10 — Define what success looks like and publish performance data

Ask:

• Where are completion rate, user satisfaction, digital take-up, and cost per transaction published, and how fresh are they?
• What is user satisfaction this quarter, and why did it move?
• Which step's drop-off are you investigating with research rather than guessing?
• Who is accountable for the published figures, and can they explain the definition?

Evidence to request: the public performance page or the draft that is about to be published, definitions, the satisfaction series.

Answers that indicate Met: the four measures exist, are current, and the team can explain a movement with something they heard from users.

Answers that indicate Not Met: no publication, or satisfaction with no completion rate. Red line L5 on the publication side.

Do not rerun Beta

If the team offers the Beta usability report as their Live evidence, ask what they have learned since. The Beta report can explain the starting point. It cannot close point 1.