Embedded Files
Thesis Project:
Visual Conveyance in a Physically Constrained Space for Mobile Augmented Reality Adventure Game Levels
PROJECT DETAILS
Role: Interactive Experience Designer
Engine: Unity & Google ARCore SDK
Platform: Android
Development: 4 Months
Completed: 2019
Team Size: Solo Project
Assets: Synty Studios
INTRODUCTION
In 2017, Google and Apple announced the release of their respective augmented reality (AR) platforms to the latest smartphones [1]. As developers raced to take advantage of this new technology, they ran into their first hurdle: AR has changed the way we interact with mobile games.
The game is now beyond screen-space, and with the player in control of the camera, it’s difficult for designers to decide how best to guide the player through a level. If the player is too far away or at the wrong angle an object's touch target is too small, or it may be hidden behind another object.
How do we get players to go or do what we want them to? We teach them.
A level designer can teach a player how to play a mobile AR game without verbal or written instructions, using only visual conveyance.
To demonstrate this, I created a level for a mobile AR game I named RushAR.
RESEARCH
The research was conducted in three areas:
Visual Conveyance
Adventure Game Levels
Mobile Augmented Reality
From each area, I compiled a list of best practices.
Visual Conveyance
Visual conveyance can be broken down into two elements: signifiers and affordances. A signifier, as Don Norman defines it, is “Any mark or sound, any perceivable indicator that communicates appropriate behavior to a person” [2]. They teach us what the affordances of an object are. What can it do for us, or to us? Can we use it, or will it hurt us?
Without signifiers, an affordance will remain unknown, unless we are specifically told it exists [3]. Having too many signifiers can overwhelm a player, effectively hiding affordances as well.
Visual Conveyance Best Practices
No Wordy Instructions
Use Signifiers to Teach Affordances
Signifiers Need to Make Sense
Don’t Overwhelm the Player
Adventure Game Levels
Puzzles are a common element in adventure game levels. An adventure puzzle needs to be easy to identify, so players do not become frustrated wondering what they’re supposed to do next [4]. By maintaining theme continuity players know what is and what is not part of the puzzle.
Adventure puzzles often don’t come with instructions. Players learn the affordances of the puzzle through exploration and discovery, and they do it at their own pace.
When researching games, such as The Room, Faraway Puzzle Escape, RIME, the board game Rush Hour, and escape rooms I looked for their use of common knowledge, positive and negative space, shapes, color and contrast, and movement to signify the affordances of their puzzles.
Adventure Game Levels Best Practices
Proper Signifiers are Critical
A User-Controlled Pace
Puzzle Needs to be Easy to Identify
Theme Continuity
Exploration
Mobile Augmented Reality
Google is arguably the first to create a clear set of mobile AR design guidelines [5]. Their well-illustrated documentation is updated regularly, sharing the knowledge they’ve gained through their own research.
Interactive objects need to be easy to identify, using “…color combinations, glowing outlines, or other visual highlights.” When an object is far away from the player, it’s touch target may become too small and the player’s gesture may not be detected. Due to this, it is important to “Provide reasonably sized touch targets despite the object’s small size.”
Designing an experience that encourages exploration can naturally ease players into moving around while playing. If you have them moving too fast though, ARCore will lose its tracking information, breaking immersion as the content slides away into infinity.
From both Google and Pokémon Go, we learn the importance of keeping players safe, defining an experience size, preventing them from wandering away looking for content that isn’t there.
Mobile AR Best Practices
Easy to Identify Interactive Objects
Reasonably Sized Touch Targets
Encourage Movement
Keep Track of Tracking
Define an Experience Size
ARTIFACT
RushAR is based on a popular board game, Rush Hour. It was developed using the Unity game engine with ARCore SDK for an Android smartphone. The 3D game assets used are from the Unity Asset Store.
In designing RushAR I condensed the best practices from my research into three guidelines:
Use Proper Visual Signifiers to Teach the Player the Game’s Affordances
Ensure Steady Tracking by Encouraging Exploration
Provide Reasonably Sized Touch Targets
To gather data on this project I presented it to 32 participants with the following rules:
Participants can be offered help during the AR setup phase if needed.
No information about the game itself can be given until after they complete the survey.
Do not tell them they can move around during the experience.
After they played the game, they were asked to complete an anonymous survey using the Likert Scale.
AR Setup Phase
AR setup can be overwhelming, so I separated the information out into steps:
A hand animation and a toaster message to instruct the player how to move the device in a way that helps ARCore recognize and render feature points and flat surfaces in the world.
A flattened outline of the game with a 3D title as a placeholder to indicate its position on a flat surface and communicate the experience size.
A final step to allow rotation of the game before it is ready to play.
To determine the effectiveness of RushAR’s setup phase, I asked three questions:
How familiar are you with mobile devices?
How familiar are you with augmented reality?
How clear were the on-screen setup instructions?
All 32 participants were familiar with mobile devices. A majority of them were familiar with augmented reality at some level. And almost all of them understood the setup instructions.
Teaching Through Visual Conveyance
The objective of the game is to get the red car past the traffic jams and out through the construction cones at the other end of the level.
To establish the importance of the red car, all non-active vehicles were desaturated, and the red car was set by itself in a less crowded part of the map. Due to red and grey being difficult to differentiate for those with color blindness, I added 3D arrows above all movable active cars with animated textures.
Once moved, the red car activates the rubbish truck and the blue car, creating the tutorial area of the level. A natural desire to move these obstacles out of the way increases the player’s investment in the red car.
Out of the parking lot, the red car turns down the next street. This communicates to the player a constraint: they don’t have complete control of the car.
The next two puzzles establish a pattern and reinforce the player’s expectations of the game. When the last puzzle is complete and the car drives through the cones towards the edge of the map, the player knows they can’t stop it and conclude that the objective has been reached.
To determine whether the visual conveyance in RushAR effectively teaches players how to play it, I asked four questions:
Have you ever played the game Rush Hour?
How clear is the objective of the game you just played?
How easy were the controls?
Did you complete the game?
Out of the 32 participants, 84% of them understood the objective well enough to play the game. 88% of them found the controls easy enough to use. And 75% of them played the game to its conclusion.
Encouraging Players to Move
Getting players to move the device in and around the game was a challenge. Early prototypes of RushAR used blocks instead of cars. In every Kleenex Test, players did not move at all.
This presented a problem with what Google refers to as “touch target proximity”. Touch targets in RushAR overlapped each other when their screen-size was fixed, and matched-size targets became too small at distance.
I hypothesized that by using art assets more interesting than blocks, players would naturally want to move in closer to explore. This led me to acquire assets from the Unity Asset Store.
To determine whether the final version of RushAR was effective at encouraging players to move, I made one statement and asked two questions:
I believe the pieces were too small.
How often did you change where you were standing while playing the game?
How often did you move the device closer to the game?
59% of the participants thought the pieces in RushAR were too small, however, 81% of them changed where they were standing during play, and 97% of them moved the device closer to the game while playing it.
LESSONS LEARNED
A Stronger Hero
During the study, I received excellent feedback from colorblind participants: use an emergency vehicle instead of the Red Car. Flashing emergency lights and a silhouette that naturally stands out is a better use of visual conveyance than an animated 3D arrow. While the Red Car honors the game’s Rush Hour roots, this change would be wholly appropriate for RushAR’s theme and the goal of this thesis.
Leveraging Psychology
It is well documented that humans are attracted to disasters because of our survival instincts. The knowledge we gain from witnessing such events is valuable, and instinct pulls us to the edge of danger.
An emergency vehicle as the hero invites an obvious story to the level: an accident. Instead of traffic cones leading off the edge of the map, I would instead place the remains of a recent collision. A small fire from one of the vehicles would produce a column of smoke rising above the city. A diegetic sign pointing to the objective.
Such movement and spectacle are more likely to draw players in than mere sightseeing. Add tiny people to the scene, staring with morbid curiosity, and the player finds themselves joining the crowd, as they might do in real life.
Beckoning Them In
Add characters to the scene that react to the player’s proximity. Have them turn, look up at the player, and beckon with their little arms and hands to move in closer. At a specific range, the figures can then turn and point to important objects.
Have a special character guide the player through the level. In the game RiME, solving the first puzzle summons a fox that runs towards objectives and even sits on key puzzle pieces, wagging its beautiful tail. If the game recognizes the player might be stuck, the guide can come right up to the camera, and like Tinker Bell in Peter Pan, waggle a disapproving finger at them before harrumphing and zipping off, leaving a trail of pixy dust to follow.
If they wander out of view, it’s helpful to have an off-screen indicator, such as a 2D carrot in screen-space, to help the player find them again.
While Android’s mascot, sometimes referred to as Bugdroid, is free to use on their platform, a theme-appropriate option for RushAR would be a news or police helicopter with a spotlight to point with.
Easter Eggs Reward Exploration
Adding a secondary objective to the game will reward player exploration. To achieve this, I would place a collectible treasure in a location easy to see from most angles and reveal a 2D counter in screen-space after its collected.
RushAR has vignettes: an auto repair garage, police have cordoned off the alley behind the pawnshop, and deliveries are being made to small shops. I could also create a mini-game by adding a diegetic photo-op sign, a script that makes a camera button element appear when the player is at the right distance.
Choosing a Toolset
Development tools are constantly being updated and careful consideration is recommended. Two toolsets were used during RushAR’s production: Unity’s AR Foundation, a cross-platform API that provides helpful solutions to common AR development problems, but depends on a large collection of independently updated plugins; and Google’s ARCore SDK, currently the only option with rapid iteration through Instant Preview, but requiring some content to be severely scaled down.
Out of the box, many toolsets present challenges with features like physics and scale. ARCore, for example, works by translating content to an anchor point, a problem if you need terrain, which currently can’t be moved at runtime. The content purchased for RushAR had to be scaled down to a 1:285 ratio, which produced problems with collisions in Unity’s physics engine.
CONCLUSION
Level designers can teach players how to play mobile AR games without verbal or written instructions, using only visual conveyance, as I have shown in RushAR. The adjusted guidelines I used and the lessons I learned, provide a good starting point for future research and development.
REFERENCES
[1] R. Amadeo, "Google's ARCore brings augmented reality to millions of Android devices," 29 August 2017. [Online]. Available: https://arstechnica.com/gadgets/2017/08/googles-arcore-brings-augmented-reality-to-millions-of-android-devices/.
[2] D. Norman, "Design of Everyday Things," in The Design of Everyday Things: Revised and Expanded Edition, Basic Books, 2013, p. 14.
[3] J. J. Gibson, The Ecological Approach to Visual Perception, 1977.
[4] "Designing and Integrating Puzzles in Action-Adventure Games," Informa PLC, 6 December 2002. [Online]. Available:https://www.gamasutra.com/view/feature/131326/designing_and_integrating_puzzles_.php?page=3
[5] "Augmented Reality Design Guidelines," [Online]. Available:https://designguidelines.withgoogle.com/ar-design/
[6] Samsung Electronics Co., LTD, "Samsung.com," [Online]. Available: https://www.samsung.com/global/galaxy/galaxy-s9/
[7] "Augmented Faces," [Online]. Available:https://developers.google.com/ar/develop/unity/augmented-faces .
[8] "Augmented Reality - The Past, The Present and The Future," September 2019. [Online]. Available:https://www.interaction-design.org/literature/article/augmented-reality-the-past-the-present-and-the-future.
[9] "No More Tutorials! How to Convey Information Through Design," 25 March 2019. [Online]. Available:https://remptongames.com/2019/03/25/no-more-tutorials-how-to-convey-information-through-design-repost/
[10] "Pokemon Go," [Online]. Available:https://en.wikipedia.org/wiki/Pok%C3%A9mon_Go .
[11] "Share AR Experience with Cloud Anchors," [Online]. Available:https://developers.google.com/ar/develop/unity/cloud-anchors/overview-unity .
[12] Unity Technologies, [Online]. Available:https://unity.com/
[13] Unity Technologies, [Online]. Available:https://assetstore.unity.com/
[14] Vision Online Marketing Team, "What is Visual SLAM Technology and What is it Used For?," 15 May 2018. [Online]. Available:https://www.visiononline.org/blog-article.cfm/What-is-Visual-SLAM-Technology-and-What-is-it-Used-For/99
[15] Egoraptor, "Sequelitis - Mega Man Classic vs. Mega Man X," 31 October 2011. [Online]. Available:https://www.youtube.com/watch?v=8FpigqfcvlM&feature=youtu.be
[16] T. Kelly, "Conveyance," [Online]. Available:https://www.whatgamesare.com/conveyance.html .
[17] M. Potuck, "Apple announces ARKit for iOS 11," 5 June 2017. [Online]. Available:https://9to5mac.com/2017/06/05/apple-announces-arkit-for-ios-11/ .
[18] P. Reynolds, "Google's AR design guidelines: baby steps - but a good starting point," 28 July 2018. [Online]. Available:https://thenextweb.com/contributors/2018/07/28/google-ar-design-guidelines-bad-good-2d-3d/ .
[19] "Augmented Enviroments Georgia Tech," [Online]. Available:https://www.youtube.com/channel/UCkqXtx9nhZ7w4zfuQaR296g .
[20] "Experience Size," Google, [Online]. Available:https://designguidelines.withgoogle.com/ar-design/environment/experience-size.html# .
[21] "3D User Interface Elements/Animated/ Low poly," BrainBox, 8 February 20126. [Online]. Available:https://assetstore.unity.com/detail/3d/props/3d-user-interface-elements-animated-low-poly-13538 .
[22] "Affordances - How Design Teaches Us Without Words," Extra Credits, 29 January 2014. [Online]. Available:https://www.youtube.com/watch?v=QCSXEKHL6fc .
[23] "Simple Town Cartoon Assets," Synty Studios, 14 May 2019. [Online]. Available:https://assetstore.unity.com/detail/3d/environments/urban/simple-town-cartoon-assets-43500 .
[24] "The Evolution of ThinkFun's Rush Hour," ThinkFun, Inc., 1 February 2018. [Online]. Available:http://info.thinkfun.com/stem-education/the-evolution-of-thinkfuns-rush-hour .
Page updated
Google Sites
Report abuse