User Testing Guide
Overview
User testing and user experience research is about observing how people interact with media.
Author: Bill Fischer
Low fidelity Testing
The image above shows middle school students engaging in a low fidelity test of a mixed reality design using cardboard prototypes and taped-off spaces. Much can be learned from these types of tests during early design phases.
Testing the Amplify: Journey To Mars Virtual Field Trip concept (external link) as part of The Epic Project.
High fidelity Testing
The image below shows an in-class test of a board game that is in its final design stages. The production values are at their highest, and the several rounds of preliminary test were completed during earlier design stages.
Testing the Minor Madness Code Theory board game (external link) as part of The Epic Project.
Testing
It is important to not interject yourself into the testing. Do not give the user any introduction to it. Simply put your media in front of the user and let them interact with it. Prompt them only if:
They are completely stuck and cannot move forward
You need to observe them interacting with a part they have not yet visited
You need them to attempt to accomplish something very specific
Don't Do this!
Observing
The user experience involves more than just the human / machine interaction. It includes the environment, the user demographics and events directly before and after the test. Photographs are a great way to capture the peripheral user experience. There is also software that can track navigational history and eye movement for one to one interactions with media. Here is a list of observations that should be included in a user test report.
Name of the Media
Date of the test
User Demographics (age, education, occupation, hobbies (if applicable) number tested
Description of the environment where the test took place (lighting, noise, seating, temperature, etc.)
Description of the technology used (computers, projectors, displays, devices, etc.)
Description of events users were engaged in just prior to the test
Description of the User Experience with the media. Record:
Points of hesitation or confusion
Questions the user was compelled to ask
Emotional reactions at various points (confusion, joy, accomplishment, etc.)
Readability / view-ability of the media
Description of media efficacy (how well was the user able to accomplish the goals of the media)
Description of user behavior directly after the test
Dialogue
After the observational period, you may engage in dialogue with the test subjects. They are usually not design professionals and will not have the language or the framework for providing feedback in an articulate and useful manner. They will use useless terms such as "good", "bad" and "like". If you find yourself in a dialog with the user, prompt them to provide feedback in these categories:
Does the design function the way they would like it too?
How does it make them feel?
What is memorable, for them, about the design?
Often they will try to describe a problem by providing a solution : "why don't you make that blue?". What you should gather from that comment is that they think the color is not working. You need to probe them until you can find out why they think the color is not working, then choose a color yourself that solves that problem.
Summarizing
A summary brief should be completed to provide an overview of the most important points to be addressed by the next design phase. here is an example:
"The common theme throughout the interviews was that immersive experiences are critical to the overall quality of the field trip. The negative moments were often the most memorable because they added spontaneity to the experience. Safety is a big concern among teachers and chaperones. Transportation and travel could be frustrating at times depending on the length of time. The smaller the group size, the more effective the field trip is."
Teamwork
Here is an example of how a team can work together.
Interviewer
Greets the interviewee.
Tell the user what is going to happen and how
Gets the process started by asking an open ended question
Answers Questions
Controls the timing
Note Takers / Observers
Be unobtrusive and quiet
Type the notes into your computer or phone
Include your own observations about how the subject is feeling in relationship to their experience
Photographer
Try to capture expressions that reflect the users feeling toward the media
Ask the interviewee to pose for you. Have some fun with it.
Pay attention to composition, lighting and framing. Make them interesting.Photography Assistant (1)(optional)
Carry extra batteries
Location Coordinator
Scout locations as soon as you arrive.
Make sure the "set" is dressed properly and change if needed.
Alter the lighting if needed.