Chapter 3: Collections, Setting, & Interpretation in a Metaverse Environment
Collections
As we attempt to create more effective and immersive virtual museum experiences in the metaverse or otherwise, an important element to prioritize is objects and collections. In physical museum spaces, objects are the bread and butter of what museums are all about. While we can look beyond this classic definition in the metaverse, museums shouldn’t disregard the strengths of their collections in favor of something dazzling but without substance. Collections link us to our past as human beings or help us learn about the natural world. Since we center our museums around objects and collections in the physical world, why should this be different in the metaverse? Objects also help visitors interact with each other in museum settings: “Social objects are the engines of socially networked experiences, the content around which conversation happens. Social objects allow people to focus their attention on a third thing rather than on each other, making interpersonal engagement more comfortable” (Simon, 2010, pg128).
In the physical world, we in museums put a lot of focus on our collections being "the real thing." How do people interact with “the real thing” in museums? Is “the real thing” always necessary for visitors to have meaningful museum experiences? Researcher and professor K.F. Latham’s 2015 study and subsequent report on these questions explore some interesting takeaways. Latham breaks down a visitor’s experience with “the real thing” in museums into four main themes: presence, self, relation, and surround.
So, when sharing these collections in the virtual space, efforts should be made to recreate the effect they have ‘in real life’. One participant in K. F. Latham’s study explains that specific details of an object were what made the experience the most real and impactful for her. The participant, Melissa, discusses a perfume bottle in an exhibit about the Titanic shipwreck: “I could see the liquid. . . I could see the liquid right in front of me. It wasn’t just a picture. I could see that right there” (qtd. in Latham, 2015, p.10). Latham further explains, “The fact that [the participants] can ‘see’ it (meaning, be with it physically) . . . is crucial to the experience. It is a physical thing that is near me, in the same space with me . . . and it was also in the past at some other time seeing things in the past, conflating the then and now” (ibid). How can we recreate this effect in a metaverse museum?
One approach is realistic 3D-modeling, which is using software to create digital, interactive models of real-world objects. One of the benefits of 3D models is that they allow interactivity well beyond what would be possible in a physical museum due to conservation concerns. In 2013, the Art Institute of Chicago created a 3D model of the Augsburg Cabinet, a furniture piece made by Adam Eck circa 1650. AIC published a video (left) showing an animation of all the drawers and doors of the cabinet opening and closing (Schmidt, 2012). The textures, carpentry, and sculpted details of the original cabinet are all rendered in such detail that at times it is difficult to tell that it’s an animation and not “the real thing.” However, an object in the metaverse need not be ultra-photorealistic: “virtual world realism is not purely additive nor, visually speaking, strictly photographic: in many cases, strategic rendering choices can yield better returns than merely adding polygons, pixels, objects, or bits in general” (Dionisio et al., 2013, p.34:9).
The YouTube clip on the right shows one such example from Sherlock Holmes: The Devil’s Daughter (2016). When the player (as Sherlock Holmes) clicks on certain points of the pocket watch, Sherlock verbally states details that he notices about the watch. These lines of dialogue are captioned, increasing the accessibility of the object interaction.
Similarly, a museum could create a 3D model of an object such as a sculpture, and design specific points of interest on that sculpture. Label text that would normally be displayed on the wall could be read aloud (and captioned) depending on where the user clicks, taps, or gestures. “The potential to handle virtually and to scrutinize these [objects] as 3D models, using fingers on a touch screen or a mouse, simulates the experience of the original owners, who may also have held, observed, and rotated [the objects] in their hands” (Jeffs, 2020, pg317).
Setting & Interpretation
A state-of-the-art virtual world should be immersive. “The environment’s level of spatial, environmental, and multisensory realism creates a sense of psychological presence. Users have a sense of ‘being inside,’ ‘inhabiting,’ or ‘residing within’ the digital environment” (Dionisio et al., 2013, p.34:5). Most virtual museum experiences thus far have been somewhat passive, where a virtual visitor scrolls through a “virtual exhibit” that is more like a regular webpage article than anything that they can walk through. Some virtual exhibits use platforms such as Matterport, where a spherical camera captures real-world spaces and turns them into a 360-degree walkthrough, like a Google Street View for interior spaces. Museums can add nodes of interest in Matterport and copy in their label text from the exhibition for users to read.
However, one of the main elements missing from these versions of virtual exhibits is the social element. Visiting a museum with other people is something we take for granted in physical galleries. Even visitors who arrive solo can still interact with museum staff, docents, and strangers that they encounter in the museum. Visitors can strike up conversations with each other around a work of art or help each other win an interactive exhibit game. The metaverse seems to be offering a way to increase virtual social interaction for museums. “Cultural institutions don’t need to offer every kind of interaction under the sun – they just have to pick the few interactions that most support the kind of behavior and content creation that they value. There’s power in the specific decisions about whether users will be allowed to contact each other directly, make comments or ratings, or produce various kinds of digital and physical artifacts” (Simon, 2010, pg122).
One participant response in K.F. Latham’s study on “the real thing” in museums illustrates this idea. “Helen,” an 80-year-old study participant, stated that since she is afraid of traveling over open water, she will never be able to travel to Rome. But looking at a painting at the Columbus Museum of Art, Cascatelli, Tivoli, Looking Towards Rome, she shares, “I’ll never be there to see that in person, but I could stand there imagining myself going up the mountain” (qtd. in Latham 2015, p.7). If a painting can help transport visitors to another place or time, could a metaverse museum have a similar or even stronger effect?
Helen’s comment about traveling reflects the situations of many visitors who are unable to travel to different locations due to many reasons. But a metaverse museum experience could allow people from all over the world to log into a virtual tour with their avatar or use a Virtual Reality headset to help with increasing the feeling of “being there.” If Helen were to view this painting virtually in a 3D environment, would it be enough for her to walk down halls of a virtual gallery? Why not use the painting as a 3D portal to Tivoli to create the closest thing to a “real” journey to Italy?
“Evidence to date suggests that if our objectives in creating these virtual worlds is to provide a means of understanding structure and layout, to experience the feel of a place and a sense of the environment, then the ability to actively navigate and explore the world rather than passively observe it” can help us achieve these objectives, writes Jeremy Huggett in his 2020 article “Virtually Real or Really Virtual: Towards a Heritage Metaverse.” If we build a museum experience within the metaverse, what do we want our audiences to experience? What do we want them to feel, see, and do? What possibilities does the metaverse offer for interpretation that the real world cannot?
Creating an immersive setting within the metaverse can help offer new or renewed interpretation of museum collections. The National Museum of Korea’s new metaverse experience in largest Asian metaverse platform, Zepeto, is a good example of this. Peaceful Hill, launched in October, 2021, “exists with two representative collections of the National Museum of Korea, Pensive Bodhisattvas. They stand in a peaceful natural environment filled with light, not like in real world where they are normally displayed indoor space. Visitors can stroll around the map, brighten the Buddha statue by finding hidden gems and take a selfie with the statue inside mysterious cave. Visitors can also climb up the trees, sit on rocks for meditation and chat with friends on a grassy field” (Metaverse Museum “Peaceful Hill,” n.d.). The statues in Peaceful Hill are ones that are normally displayed indoors, protected from the elements in a museum gallery environment. But the metaverse experience offers visitors a way to view the statues in a virtual environment close to how they would have been installed in nature. “Presence goes beyond the individual’s sense of ‘being there’ within the virtual environment. It can also relate to the presence of social others (real or virtual agents), and the responsiveness of the environment to the individual (their ability to act in and on the environment)” (Huggett, 2020, pg4:5).
With so few examples of museums creating metaverse experiences so far, it’s difficult to also find examples of how museums are interpreting their objects. Although it was not created by a museum, Animal Crossing: New Horizons’s in-game museum has didactic material for the works of art that players can collect and donate to Blathers, the museum curator owl. Players also can catch insects and fish and dig up fossils, but the game doesn’t provide didactic text for bugs, fish, and fossils other than naming them. However, when the player gives any of these creatures/items to Blathers, he will ask if you would like to hear a factoid about them. The works of art, on the other hand, have purely textual labels that appear when you interact with the artworks in the gallery. The artwork included in the game was sourced from real-world museums such as the British Museum and Metropolitan Museum of Art, so it stands to reason that the interpretive label that appears is from their respective institutions, but I could not find a specific source to confirm this.
As for how to design and prepare object interactions in a metaverse or other immersive virtual environment, museums can look to existing video games for ideas. A genre that designs object interactions particularly well are detective and mystery games, such as the Sherlock Holmes series by Frogwares, a Ukrainian game development company. Since much of the gameplay of this genre depends upon the player finding clues by examining items, recent titles take advantage of the capabilities offered by Unreal Engine and Unity to allow players to manipulate objects by rotating them, pushing buttons, opening them, and closing them.
Programs
The metaverse has particularly exciting opportunities for museum programming and tours. Virtual tours have historically meant that attendees register for a Zoom meeting and are then shown a slideshow presentation by museum docents. The Ayala Museum in the Philippines took a different approach when they created an 8-bit virtual experience using the platform Gather.Town for 2021 International Museum Day.
“The tour kicks off in the Ayala Museum’s lobby. This is where a tour guide introduces the platform to novices and shows them how to navigate around the space and move into new places. What’s more, visitors can choose the avatar’s name and alter details like clothing to personalise the entire experience. Once they are up and running, virtual visitors can move their avatar and interact with objects and signs with a simple keystroke, just like many of the best video games of yesteryear. In addition, further key commands allow visitors to turn on their microphone and camera with web-based interactions if they wish to have them” (Charr, 2021).
A virtual museum tour in the metaverse could be much more interactive and can simulate touring a museum in-person. Museum docents could also be “there” with their avatars, and a whole group of people could virtually gather in a virtual gallery or exhibit.