Lines of questioning — Beta lead

Point 6 — Have a multidisciplinary team

Ask:

• Who is on the team during private beta, and who responds when a user is stuck this week?
• Which roles disappear at public launch, and what fails if they do?
• How is the service funded for the six months after this assessment?
• Who is the civil servant who can change priority without asking the supplier?

Evidence to request: current team list, funding line, named product owner in the civil service.

Answers that indicate Met: the private beta team is the public beta team, with operations included.

Answers that indicate Not Met: a launch squad is being hired separately, or user research has already rolled off.

Point 7 — Use agile ways of working

Ask:

• What did you release during private beta, and what did you pull because of evidence?
• How long does a small change take from decision to users?
• What does the sponsor review: the service, or a status report?
• What will you deliberately not open up on the first day of public beta?

Evidence to request: release history, a cut from the backlog with a reason, the public onboarding plan.

Answers that indicate Met: the team can show a change that followed a private beta finding, and the public plan is staged.

Answers that indicate Not Met: the service has not changed since the private beta opened, or public launch is a big bang the team cannot roll back.

Point 10 — Define what success looks like and publish performance data

Ask:

• What is the completion rate, and where do people drop off?
• How will you measure user satisfaction, and what is the baseline from private beta?
• What counts as a digital completion versus an assisted or paper completion (digital take-up)?
• What is inside cost per transaction, and who will publish the four measures?
• Which measure would tell you public beta is hurting users?

Evidence to request: KPI definitions, a baseline from private beta, drop-off by step, the publication owner.

Answers that indicate Met: all four of completion rate, user satisfaction, digital take-up, and cost per transaction have a definition, a baseline, and a place they will be published. Drop-off is shown for the worst steps.

Answers that indicate Not Met: the team reports applications started, or the dashboard is planned for after launch.

Volume

Ask how many people used the private beta and whether that included the cohorts user research said mattered. If the number is large and the cohorts are narrow, say so in the narrative and leave point 5 to the user-research assessor.