Embedded Files
Role:
- UX Researcher
- UX Designer & Developer
Background
Background
Transformative Play Lab (UC Irvine) and Apples and Oranges Arts collaborated to organize the course, ‘AR and VR in Theater’. The aim was to explore how augmented and virtual reality (AR and VR) technologies can be leveraged in theater plays and musicals.
Storytelling has taken different forms through the centuries and remains an important part of our culture. What can immersive technologies like VR offer? We worked on the theatrical play assigned to us - 'The Next FairyTale' (TNFT), which was scripted by Brian Davis and produced by Apples and Oranges.
Studying current efforts that largely used 360* video encouraged us to prototype a similar solution for (TNFT). However, we believed that we were not using VR to its full potential. We wanted to design an immersive and engaging experience that would allow people to interact with characters and feel connected to the story.
In our system, viewers enter the VR experience as one of the characters from the play. Our goal was to support the viewer's transformation and role play through VR, and help them learn in the VR environment.
Figure 1
Figure 2
Figure 3
Research & Design
Research & Design
We designed the VR experience with the goal of using it as pre-show/post-show experience when TNFT actually plays in theaters. We chose to build our world using voxel art to create the impression of a fairy tale world.
In our experience, the user takes on the role of Calliope, a Fairy Godmother. They encounter Minerva, the Queen of all Fairy Godmothers, who also happens to be Calliope's sister. Minerva is in a dilemma and discusses her problem with Calliope. Players, as Calliope, can influence the emotions in the conversation as Minerva tries to make a decision, using different emotion spell balls (Happy, Calm, Sad and Angry - Figure 2). Different spells evoke different reactions from both Calliope and Minerva.
We designed it to be a narrative-based role playing experience and our design was informed majorly by research in the areas of role play in games, learning in games, theatre and acting.
User Study
User Study
Research Questions
Research Questions
We conducted a user study with 12 participants to evaluate this VR experience. We were interested in the following research questions:
- What role do players see themselves in?
- What are the interactive possibilities in a VR experience where the player takes the role of a character in the play?
- How do we design for learning in VR with respect to role play?
- What role does VR play in the experience? What does it augment? What is lost?
Methods
Methods
- We gave a brief introduction about the original play on which this VR experience was based, and the VR experience itself to every participant.
- We also provided a quick tutorial in VR for the controls and teleporting functions. The participant underwent the experience after they felt they were comfortable and ready to do so.
- We encouraged the participants to think out aloud during the experience and took notes.
- We also captured screen video of the entire experience and took observation notes.
- After the experience, we conducted a semi-structured interview (audio recorded) with the participant which lasted about 30-45 mins.
Analysis
Analysis
As a team of three, we transcribed and coded interviews. Each of us also went through other interview transcripts and codes, to ensure reliability.
For the first round of qualitative coding, we read through interviews and our observation notes, and coded phrases of a sentence or whole sentences. We added our codes, quotes from the participant, and relevant tags in a shared Google Sheet. We would check if any existing code applied before adding a new code. We also added additional notes and observations from interviews as analytical memos.
In our second round, we went over our descriptive codes and started grouping similar ones together. Thus we iteratively generated categories and sub-categories of our codes.
We mapped our final codes and categories onto the white chart for brainstorming higher-level themes and relationships between code categories through affinity diagramming (Figure on the left)
Findings
Findings
We detailed our findings and design recommendations in a full-length research paper. The following section is a brief summary of key findings and design implications.
Roleplay
Roleplay
We found that our participants were spread across a spectrum with respect to role playing. Some of them felt like a player in a game while few got into the skin of the character.
1. We observed that VR environments both supported and hindered role play. A few participants were distracted by the novelty of the VR environment and missed paying attention to important aspects. However, the 'uncanny valley effect' (observed in Figures 1,2 & 3) helped participants to leave their own world behind and enter the fairytale.
2. The Mirror enables the players to see themselves as Calliope, and hence role play the character. Apart from the visual transformation, it makes them feel like they have physical form in VR. However, a few participants refused to suspend disbelief. For example, one participant said, "I don't wear a dress, I don't think that's me..".
3. Agency over Calliope's actions and emotions were strongly tied to how much they felt like Calliope. A perceived lack of agency also led to participants feeling like players in the system.
Social Interactions in VR
Social Interactions in VR
Social interactions in VR mirror real life interactions! Despite Minerva being a block figure, participants treated her like a real person.
---> We found that participants drew norms from real life while interacting with Minerva.
- They were concerned about standing too close to her and wondered who would begin the conversation. T
- hey choose spells in attempts to respond sensibly to her troubles, but they also expressed concerns over manipulating her emotions.
---> But this went beyond typical social norms.
- People (role playing) were mirroring real life conversations to the extent of caring for Minerva, being sensitive to Minerva, and her feelings.
- Few users admitted that they felt responsible for her. We can see this in the way they used different emotions. Participants used happy and calm emotions to ensure she was not depressed, and negative emotional spells to make her reflect and express frustration.
Learning in VR
Learning in VR
Learning is important in these experiences as we realized through our analysis.
Participants needed time to understand what spells are for and their effects to make meaningful decisions. Minerva’s dress change in response to spell selection was quite prominent and was picked up by most of the participants (See Figure 2 & 3). Decision making was exploratory in the start since participants randomly picked up spells to see the effects. Some even wanted to try them out on themselves first to learn how spells worked.
Few participants felt overwhelmed by the experience due to several reasons such as being new to VR itself, this particular VR space, learning new interactions and so on.
We found evidence of two types of learning - exploration-based and modeling-based learning as detailed by Ray.
Some participants wanted to explore the environment and understand the experience through several interactable objects, while some preferred the instructional method of being told what will happen next.
We observed that participants wanted to explore since it is VR - The important question that arises is how to balance freedom and constraint and remind them of their goal. Finding that fine balance between these two types of learners is crucial while designing theatrical and storytelling experiences generally in VR.
Design Recommendations
Design Recommendations
To lend structure to the way we would think about design recommendations, we did this -
Our team of three, including me, did a design exercise. We decided to sketch this VR experience from the scratch. Now that we had the knowledge we had from all this data and findings, how (differently) would we go about designing it now? What changes would be make? What design elements would be improvise or remove?
We then shared our designs, discussed our thoughts, iterated over them, and generalized them.
This exercise was immensely useful, and fun as well!
We derived design implications for designing VR systems that have story-telling experiences and provide role play capabilities and some level of agency to the user/player. We have detailed them here - our research paper that was accepted at ACM CHI 2020.
Conclusion
Conclusion
Transformative experiences that bring about empathy can be powerful and can be incorporated in HCI design and practice. Even if VR experiences are not always the way to achieve that, we hope this study illustrates the interactive possibilities of role play in VR and its transformative experience, as enhanced by VR.
Page updated
Report abuse