Prototypes & Testing

We conducted two rounds of prototypes, each followed by a user testing session. We have included prototype visualizations (one is annotated), testing session descriptions, main findings, and revisions.

Prototype 1

We used a paper prototype for Testing I. We took photos of main features of our paper prototype and created an annotated slide show, then created a video.

Prototype 2

We used a revised paper prototype for Testing II. Above, we converted our paper prototype into digital form and created a video of the main features.

Testing I Description

Setting

Two testing sessions have been held in Peabody Hall.

Participants

Five testers (MEITE students) who are not our target learners tested paper prototypes, were assessed for learning objectives, and provided design feedback. In the first round, tester J and tester S did the test together, but we separated them later. Then a one-to-one test was conducted on the testers.

Tasks

To begin the testing sessions, we provided the background information of traditional Chinese classrooms to testers.

The team members were responsible for observing the tester and taking notes about the tester’s experience with the prototype. Once the tester finished exploring the features in ASK VR, one of our team members asked the following Interview Questions:

1. What would you describe as the main feature of our prototype?

2. What do you think was the learning goal?

3. What were the features you liked or didn’t like?

4. What are some features you think would be helpful?

Testing II Description

Setting

The second testing was conducted in one of team member’s apartment and testing process was one by one.

Participants

Three Chinese international students who were our target learners completed the second testing. Only one of them had studied abroad.

Tasks

Are testers more willing to ask questions in class after experiencing ‘Ask VR’?
Whether the 'Ask VR' can help target learners identify the appropriate time to ask question in class?
Are general tips useful for testers?
What tip(s) will tester use in their classes?
After participating in the VR, what would tester consider the most helpful feature of the experience?
What do testers think of Ask VR and what features would they like to see?

Data Collection & Analysis

We provided a pre and post survey to testers. According to the post survey, we got many useful feedback.

66.7% testers agree that 'Ask VR' help them identify the appropriate time to ask questions in class.
33.3% testers didn’t sure that that 'Ask VR' can help them identify the appropriate time to ask questions in class.
100% testers are more willing to ask questions in your class after experiencing ‘Ask VR’.
33.3% testers feel very comfortable using the tips in their classes.
33.3% testers feel less comfortable using the tips in their classes.
33.3% testers feel comfortable using the tips in their classes

Main Findings I

The different voices and speakers in the testing session made it unclear with whom the user should interact—the “wizard” or “Xiaoyu.”
It was unclear to users when to proceed, particularly when time to interject their questions and gain automated feedback.
Some of the buttons confused the users. Specifically, they were unsure of how to distinguish uses of the pause, stop, re-record, record buttons.
Users struggled to understand the reasoning behind the feedback they were given; they wanted to see more explanation about why their question was “inappropriately timed” or could “use improvement.” The users also didn’t know what to do with the feedback they were given; some tried the loop a second time but didn’t know how to change their responses.

Main Findings II

We noticed our cultural bias after the testing. The first tester wasn't sure if she could interrupt the lecture anytime as she wanted.
Testers mentioned that some of the general tips were not very helpful or contained ideas they already knew.
When testing, some buttons remain unclear. Testers cannot tell which button is for starting the VR and which button is for recording their questions.

Revisions I

A. Limited the speaker to only Xiaoyu who delivered instructions and narrated through the VR experience.

B. Improve instructions to explicitly talk about asking questions during the lecture

C. Reduce the number of buttons to pause and record. Included a more thorough explanation of how to use the buttons on the right-side panel.

D. Created more learning opportunity through offering general tips at the end. The learner can select a tip to implement in a second round of the VR experience.

Revisions II

A. We will add some instructions before the VR experience:

Add instructions before the section of asking questions based on the lecture. Telling testers clearly that they can interrupt the class at any time they want.

B. Add more tips that can be used in class and remove the tip about after class. For example, add tips like eye contact with the professor, find the momentum of teachers’ speaking pace, etc.

C. Revise our buttons with more clear instructions and reduce the number of buttons to only 3. Update their labels with more accurate descriptors.

From Paper to VR

Our two prototype user testing sessions led our team member Shiqi Tao to create an actual VR experience using Adobe XD. Try it out below!

Flow 114 Screens, Last modified on Dec 08, 2021 19:00 GMT

If you have trouble with the embedded materials above, try these links instead:

Website version: https://xd.adobe.com/view/b239df67-789d-4e9b-82c6-1bb598f779ed-e76a/
VR version: https://app.draftxr.com/vr/qJafvx

Page updated

Google Sites

Report abuse