Wizard of Oz

Wat is het?

User-based evaluation of unimplemented technology where, generally unknown to the user, a human or team is simulating some or all the responses of the system. The technique has often been used to explore design and usability with speech systems, natural language applications, command languages, imaging systems, and pervasive computing applications.

In welke (ontwerp)fase toepassen?

3: Concept generation / Early prototyping

4: Evaluatie / Verfijning / Productie

Procedure

The wizard sits in a back room, observes the user's actions, and simulates the system's responses in real-time. For input device testing the wizard will typically watch live video feeds from cameras trained on the participant's hand(s), and simulate the effects of the observed manipulations. Often users are unaware (until after the experiment) that the system was not real.

The wizard has to be able to quickly and accurately discern the user's input, which is easiest for simple for voice input or hand movements. The output must also be sufficiently simple that the "wizard" can simulate or create it in real time.

Resoureces

<Hier alles wat je nodig hebt>

Stap voor stap / How to

The basic wizard of Oz procedure involves the following steps:

1. Develop a simulated user interface for the target technology.
2. Develop a detailed test plan with the instructions for the facilitator, wizard, participants and other staff. Determine if you need to set any expectations about the simulation's "performance" so participants are prepared for sub-par performance.
3. Recruit users who meet the appropriate user profile, try to cover the range of users within the target population.
4. Prepare realistic task scenarios for the evaluation.
5. Develop a procedure where the wizard can respond to input from a participant.
6. Train the wizard.
7. Design the instructions for the study so that the participant knows that they are working with an early prototype and that performance is not "optimized" yet.
8. Conduct pilot tests to refine the procedure and give the wizard some practice. Make any changes to the procedures and test plan.
9. Ensure recording facilities are available and functioning.
10. Conduct each session. The facilitator instructs the user to work through the allocated tasks interacting and responding to the system as appropriate.
11. Conduct a debriefing of the participants. Obtain feedback on the "performance of the wizard system". Tell the users about the wizard and explain why you couldn't tell them earlier.
12. Collate, analyze, and summarize the data from the study. Consider the themes and severity of the problems identified.
13. Summarise design implications and recommendations for improvements and feed back to design team. Video recordings can support this.
14. Where necessary refine the prototype and repeat the above process.