Articles

Usability test planning for multiple user groups

Usability testing with multiple user groups is most useful when differences between participants connect to product decisions. An onboarding journey may need separate evidence from everyday account holders, first-time users and people using assistive technology. The purpose is to identify shared failures and the conditions that create group-specific barriers.

A workable plan aligns recruitment, tasks, session logistics and analysis before invitations are sent. UCDmanager provides a free collaborative place for personas, use cases, test scripts and findings, while reusable research artefacts preserve knowledge across projects.

Define groups by need, behaviour and context

Start with the decisions the study must inform. List candidate groups through experience, goals, accessibility needs, technology access and frequency of use, then rank them by product risk. Demographics can expose inequity, but they should not replace observed behaviour.

Create a compact profile for each retained group: motivation, relevant journey, likely barriers and the outcome worth testing. Use one core task set for comparison, then add group-specific tasks only where the research question requires them. This reveals both universal problems and conditional ones.

Australian participants bring varied regional, cultural and linguistic experience. Use plain Australian English in instructions and do not assume idioms such as “no worries” or “arvo” will be understood. Comfort with informal language does not guarantee comfort with unfamiliar banking, insurance or government terminology.

Turn objectives into tasks and measures

Write each objective as an observable question: can a first-time customer complete a purchase, can a screen-reader user recover an error, and where does confidence change? Attach criteria such as correct completion, assistance requests, time on target and participant ratings. These measures provide evidence rather than a pass-or-fail verdict.

Use scenarios that resemble the service and its market. Before drafting new scenarios, review existing material against guidance on building a research insight repository, so old usability findings do not conflict with current tasks.

Pilot tasks with people outside the target groups. A task that needs ambiguous wording, hidden instructions or administrator permissions is usually a design problem. Confirm that each task has one clear goal, sufficient context and a defined endpoint.

Recruit and screen for meaningful coverage

Set quotas by group rather than only by total attendance. A formative round may use five to eight participants per high-priority group, while lower-priority groups receive fewer sessions. The sample should reflect product risk, available evidence and budget. Screen for recent behaviour, usage frequency and task-relevant barriers rather than collecting unnecessary personal details.

Mix recruitment channels. Customer panels and specialist services can reach busy consumers, while community organisations, local businesses and universities in Canberra, Adelaide, Melbourne and regional centres can supply relevant participants. Compare recruitment service options with direct outreach and confirm that every source can meet each quota.

Screening criteria:

Compensate time, travel and inconvenience in Australian dollars. Determine whether participants need their own device, a loan device or an interpreter. Schedule around time zones from Perth to Brisbane instead of assuming a late-afternoon slot is equally convenient across the country.

Session readiness:

Prepare the moderator and test environment

Build the moderator guide from the objectives, not around a preferred solution. Use neutral prompts such as “Tell me what you are thinking” and avoid explaining a button mid-session. Prepare the task script, probes, fallback tasks and debrief, then keep controlled versions in UCDmanager so facilitators deliver the same core conditions.

Choose in-person, remote or hybrid delivery according to the behaviour under study. Check local realities: some metropolitan homes have reliable NBN, while others depend on mobile data or face intermittent regional connectivity. Test the conferencing platform, screen sharing, recording permissions and backup contact method before invitations are confirmed.

Research material can contain names, addresses, account histories and health information. Apply the Australian Privacy Principles where applicable, minimise collection and separate participant details from recordings. A hosting model comparison can prompt discussion of cloud versus on-premise storage, access controls, retention and deletion before sensitive material is uploaded.

Run sessions consistently and safely

Run a pilot with one or two representative participants, then revise wording and timing without changing the measures. Balance session order across groups to reduce order effects. Standardise instructions and device conditions while allowing reasonable accessibility accommodations.

Capture completion, workarounds, emotional reactions, assistance and independent effort. Record observations rather than interpretations: “opened help twice” is stronger evidence than “was confused”. A think-aloud commentary should not replace observation, especially for quiet participants or those working with an interpreter.

For remote sessions, participants may share an office or home with limited privacy and unstable reception. Confirm suitability before recording and offer another arrangement when needed. Issue the incentive promptly after completion, with clear rules for rescheduling and withdrawal.

Synthesize patterns and reuse the evidence

Debrief each participant immediately, then examine evidence within each group before comparing groups. Use affinity mapping to cluster notes around barriers, workarounds, expectations and outcomes. Label every cluster with its source so the team can trace claims to sessions and distinguish a repeated pattern from one memorable incident.

Prioritise findings by impact, frequency, severity and evidence strength, not by the number of similar labels. A severe problem affecting a small group may outrank a minor repeated issue. Segment quantitative results cautiously when group sizes differ, and retain qualitative quotations with enough context to preserve their meaning.

Publish decisions beside the evidence: issue, affected groups, observations, confidence, owner and review date. Store stable artefacts in that repository, including validated problem statements, task wording, prior findings and superseded guidance. UCDmanager can hold active project records while the repository keeps lessons useful beyond a single release.