Articles

Prototypes versus live products in usability testing

Usability testing with prototypes gives teams a safe way to explore wording, navigation, workflow assumptions and recovery paths before development commitments harden. It is especially useful when a journey is new, several designs are competing, or user feedback could cause substantial rework.

A live product exposes participants to real content, network conditions and integrated services, producing stronger evidence about operational fit. The trade-off is lower control: a deployment, pricing change or support incident can alter a test, while recruitment and data handling become more complex.

Understanding what each environment reveals

A low-fidelity sketch mainly tests structure and comprehension; a clickable prototype can expose interaction sequences and explanatory content. A high-fidelity prototype is valuable for visual hierarchy, responsive behaviour and confidence, but polished pixels can disguise an unrealistic service model. Match the artefact to the question rather than treating fidelity as a status symbol.

A live-product test reveals how people behave around the current experience, including search habits, expectations of speed and reactions to genuine labels. Ideally, use a staging or production-like build with representative data rather than the public service. Record task success, errors, time on task, abandonment and satisfaction, then separate usability failures from content or system defects.

Aligning fidelity with the research question

Choose a prototype for early discovery, comparative design selection, accessibility checks on unfinished journeys and inexpensive iteration. Think-aloud sessions work well because moderators can probe confusion without exposing participants to financial or account risk. Multiple moderated rounds can show whether corrections survive a second encounter.

Choose a live product for regression testing, onboarding, permissions, checkout, search, support hand-offs and release readiness. Unmoderated sessions can add volume once tasks are stable, while moderated sessions explain the reasons behind observed friction. A hybrid sequence often gives the clearest evidence: test the concept, validate the build, then retest the released experience.

Neither option substitutes for analytics, interviews or expert review. Prototypes cannot prove demand for a real offer, and a live site cannot tell you whether people understood the intended model. Define the decision the study must support before selecting the environment.

Recruiting and running sessions in Australia

Australian recruitment should reflect the relevant market, language, disability profile and familiarity with the service. Remote testing can include participants in Sydney, Melbourne, Brisbane, Perth and regional areas, but connection quality may differ considerably. Check that video, audio and shared-screen tools work over each person’s NBN, fibre or 5G connection rather than assuming urban broadband conditions.

Operational details matter too. Obtain informed consent, collect only the personal information needed for the study and explain retention under the Australian Privacy Principles. Commercial email invitations should comply with the Spam Act 2003, and incentives expressed in Australian dollars should never be conditional on completing a task. If UCDmanager supports the project workflow, you can contact the UCDmanager team before finalising the protocol.

Protecting task quality and research evidence

Use neutral accounts and realistic but non-sensitive scenarios. Never ask participants to enter real passwords, card details or health information during a formative test. Prepare fallback tasks, screen sharing and a local troubleshooting contact, especially when the service depends on mobile location, SMS or a third-party provider.

Secure the test build as carefully as the evidence. Give each participant a temporary account, remove production privileges and delete exports after analysis. If a prototype sits on a new domain, ownership and routing checks described in this whois lookup guidance can help identify configuration mistakes before invitations go out. UCDmanager can keep personas, use cases, scripts and observations linked while recordings remain in an access-controlled store.

Selecting a practical testing approach

A staged live build gives higher ecological validity while retaining some control. Before a session, remove unrelated campaigns, broken links and administrative menus; an accidental page such as this casino bonus listing can derail a task and introduce avoidable risk. Verify analytics, consent notices, error logging and accessibility settings in the exact build participants will use.

The decision should follow evidence. Move from interactive prototypes to live-product testing when the concept is stable enough for realistic content and end-to-end dependencies. Return to a controlled prototype when feedback would be too costly or ambiguous to interpret in production.