Sensory Mapping & Moving through Digital Places

This page documents my experimentation with applying mapping methodology to virtual places, a practice I intend to implement in my co-development and co-teaching of a virtual worldbuilding curriculum for use with secondary students. Beyond my specific curricular aims, we are living in a pandemic moment where social distancing practices are enforced, and an unprecedented number of people are spending their time in virtual worlds (evidenced, for example, in these charts of usage of the popular gaming platform Steam over the past three months and over the platform's lifetime ("Steam chart," 2020)). This points, for me, to the necessity of attending to the materiality and affective potential of such spaces.

Multi-modal mapping has the potential to record dynamic, active, and experiential qualities of place, and can take a variety of forms (Powell, 2016). This document aims to generate a modest schizocartography, juxtaposing a variety of "alternative existential modes" (Richardson, 2014, p. 140) of digital places. In this exploration, I am the only participant. However, in moving beyond essentialist notions of the subject, and recognizing the subject as (re)cosntructed through intra-actions in/through place (Truman & Springgay, 2016), this piece aims to juxtapose at least three different subjectivities by documenting my memory-mapping of walks through virtual places, video documentation of my movement through those places, and video documentation of my face registering affect response during those virtual walks. Ideally, a future extension would invite other participants, hopefully generating a more varied schizocartography (Richardson, 2014) that could emphasize what commonalities and disparities of experiential affect arise from multiple bodies navigating these designed spaces. This may yield an understanding of how these game-like spaces/artifacts function as what Aubrey Anabel (2018) called "structures of feeling" (p. 132).

A particular quality of virtual places is the extent to which they are designed digital artifacts, entailing invitations and inhibitions that function as scripts prompting behaviors (Verbeek, 2006). This extends beyond the affordances and constraints present in physical human-designed places as fundamental material realities - e.g. the strength of gravity, the types of movement the participant's "body" is capable of, the solidity of surfaces - are all contingent on systems authored (though not always intentionally deployed) by designers. The consequent muddling of environmental and human agency that charges these environments makes their material affordances and constraints function to some extent like the cues of a dance routine. As discussed by Erin Manning (2013), choreographic cues, and the counterpoints they elicit, are less a mechanistic causal chain of a programmatic dance sequence rooted in "a stable notion of recall" (p. 5), and more a charged moment of potential activated by a variety of systemic and interpersonal attentions. In this way, the walking-through of the virtual world represents an improvisational dance where the designer is less choreographer than partner. Returning back to the pragmatics of my own curricular aims, one hope for my exploration of this method is to help students to attend mindfully and sensitively to this non-prescriptive choreographic quality of digital spaces as they both navigate and generate them in the course.

What follows are the video recording and drawn mapping of my 10-minute walks through four virtual places created by four different artists. These pieces are all examples of works that, when parsed through the lens of broader video game culture, were derisively labelled "walking simulators," a generic label that has now been adopted by some creators with a degree of pride.

Walk 1 (and 5): Secret Habitat

2020-03-30 20-42-25.mp4

My first recording of my walk through Secret Habitat, by pseudonymous artist "Strangethink," was lost due to a technical issue. I did not discover this until after drawing the above memory map. As a result, the video posted above is an after-the-fact recording of a second walk, made after the drawn map. This results in a juxtaposed pair of artifacts that don't have as neat a relationship as the other pairs in this exploration. However, rather than producing a flawed document of my walk through this space, the disparity between drawn map and recorded journey foregrounds a significant quality of place distinctive to Secret Habitat: its procedurality.

Each time it is visited, Secret Habitat generates a new landscape, populated with procedurally generated architectural forms which contain procedurally-generated galleries filled with procedurally-generated works of visual and sound art. Consequently, my drawn map works less as a spatial mapping and more as a movement between a series of black boxes representing different gallery spaces I entered. Moments of affect that stood out in my memory, and made it into my map included:

  • Realizing my movement was slowed as I moved through the shallow bodies of water scattered between the building-like spaces.

  • A moment of pause outside when I attended to the trees in the landscape, trying to determine whether they were also procedurally generated, or repeated instances of the same model.

  • Notable artworks that stood out in my memory. Was particularly struck by the olive-and-black piece marked with a star in the center.

  • My realization, after visiting several galleries, that the works within each gallery shared formal properties and represented variations on a particular systemic theme.

  • My surprise at enountering a shoreline in a space I had assumed would be boundless.

Re-watching the video of my second walk through this place, attending to my movement through it, as well as my expressions in the lower right, surfaces different responses incited by the place:

  • At 2:44 I bodily lean into the screen to get a closer look at a particular generative painting, while also moving my virtual "body" closer to it. This is the only piece that elicits this somatic response in me.

  • At 5:05, I pause to revisit the trees and try to determine whether they are generative or instances of the same prototype. My brow noticeably furrows - this aspect of the landscape is the only one to elicit this response.

  • At 6:30, my movement becomes noticeably slower, as I take more time with the visual pieces in this space.

  • At 8:12 I enter a "gallery," look at the visual pieces, but breeze by the audio works, reflecting a sensory bias.

  • At 9:04, in the last building I visit, the body of works there elicits both an 'o' shaped mouth and raised eyebrows from me - somatic responses unique to this gallery. Personally, I remember responding particularly strongly to these pieces (the olive green and bright crimson complements, in particular) during my visit to the place.

Walk 2: Gardenarium

2020-03-30 15-41-00.mp4

Gardenarium, by Paloma Dawkins, is a virtual place that invites a much more linear traversal, reflected in my drawn map which spirals upward in a single coil. The only moment this is interrupted is a space toward the middle, when the landscape breaks apart, and demands you make use of your ""body's" ability to jump. In the other virtual places explored in this exercise, jumping either isn't a verb afforded the participant, or its presence feels like a vestigial part of a "default" set of character movements in game-like environments. In Gardenarium, this jumping gauntlet feels much more like a traditional video game place, where the environment is adversarial and demands a certain acute command of the interplay between physical body, mechanical interface, and digital body to perform specific actions. The broken, falling, repeated red lines in that point of the map reflect the affective dissonance of this segment with the rest of what is a generally calming and inviting place.

Other experiences logged in the map include particular forms - drawn and animated by Dawkins - which emerge in the space as you move through it. These were noted in part because of visual interest, in part because they help serve as memory markers - each "species" of form seems native to certain parts of the place - and in part because many of them are activated by the participant's presence. A fair amount of the memorable affects in these virtual spaces for me come at moments of surprise in how the space responds to my presence. These include intentional responses scripted by the author - a field of plant-like forms unfurling as I walk through it - as well as perhaps unintentional technical gaffes - when I discover that I can walk through the walls of the small house-like structure, rendering its doors moot.

Watching back through the video, I see a variety of affect responses and types of movement not noted in my map (or noted differently in my map). The video reminds me that Dawkins placed cans of soda as a sort of breadcrumb trail to lead the participant through the place. I see when I pause to observe a particularly striking set of growths elicited by my presence (e.g. at 1:16). I notice a moment of disorientation around 3:40, when my (physical) gaze scans the screen, and leaves the screen for my keyboard, as my (digital) gaze tries to frame the environment in a way that makes traversal seem possible. During the jumping gauntlet from about 6:10 to 8:00, I see how my digital gaze turns downward at 7:41 to check my digital body's "footing." I see the large frown across my face at 7:07, when I fall from a particularly high point and lose progress (which makes salient how much "progress" is a value/feeling embedded in this place). Notably, my memory map logs five falls, while the video shows three, indicating this experience was magnified in my recollection. I notice how, around 7:30, I discover that I can make my digital body "run" by holding a button down, which allows me to jump further and eventually overcome that area.

The discovery of my potential to jump, and then to run, is given new meaning within the text of the piece by its ending, where a guru-like figure tells me that I've always had the potential to break out of my subjective position within the world and engage with it in a new way by pressing the "c" button. Gardenarium is not just about activating a place by walking through it and triggering canned animations of plants growing, it's also about uncovering and exploring unrealized potential(s) of our bodies, the digital bodies we extend into in virtual spaces, and the mechanical interface(s) between the two.

Walk 3: Proteus

2020-03-30 16-03-11.mp4

Proteus, by Ed Key, is another procedurally-generated world that opens with the player awakening in an ocean facing away from an island. Each visit to Proteus yields a different island. My map of my walk through Proteus is spatially nonlinear, but still quite traditional in that it attempts to map my particular instance of its island, and the numbered points reflect the ordering of different significant/memorable affective experiences. Like Gardenarium, Proteus responds to the presence of the player's body, but its responses feel less immediate and more inscrutable. Small interactions, like chasing a squirrel-like shape that escapes into a tree (point 9 in the map), feel like clear-cut, scripted, and cause-and-effect. But the series of events numbered 4, 5, and 6, where I walk to a standing stone on a hillside, night falls, and my interactions with some lightning-bug-like-shapes lead me to the center of a circle of stones, where I see an intense celestial phenomenon including shooting stars, auroras, and the seeming shift of the seasons, after which the island seems to have many more active entities on it, and the sun is a deeper color with visible rays. I'm honestly unsure to what extent I triggered that happening, but it hasn't happened the other times I've walked through this space with students. Other notable/memorable experiences logged on the map include the sunset at point 2 and the mountaintop with strange, inert figures at point 1.

Looking back at the video surfaces some experiences not noted in the drawn map. At 0:41, I bodily lean in and squint to better see the island at a distance. At 1:30, it becomes clear how sound is integrated into the virtual landscape, with different objects having different tracks or synthesized sounds associated with them, something I hadn't recalled at all when making my map. At 2:14, when I notice there are human-like forms at the top of a mountain, my face registers a certain confusion before I advance closer investigate - showing how this place uses curiosity to cue participant movement. At 3:22, at the top of the mountain, there's a point where I squint one (physical) eye, while training my (virtual) eye on the sun - seeing if it moves through the sky and, if, consequently, this is a place where time exists. Which foregrounds how time passing isn't necessarily a given in these designed digital places.

One small note that gives some significance to the auroras, shootings stars, and transformation that occur when I enter the stone circle later in time, is that, on rewatching, I realize that during my first time in the stone circle, when I pause for a moment, the trees around me change from pink to green. This might indicate a change in season, or some less prescriptive change in the surroundings - but it was a subtle precursor to the more significant transformation of place that attended my later passage through that area.

Walk 4: Sacramento

2020-03-30 16-25-21.mp4

Sacramento, by Delphine Forneau, elicited a number of affective experiences which merited inclusion on the drawn map. Perhaps most distinctive is the presence of three paths through the space, each terminating in a dotted line returning to the starting point. This reflects the temporally-bounded nature of Sacramento's digital place: you have about four minutes' time between your arrival by train, and the setting of the sun, at which point you are whisked back to the train tracks to depart. The ten-minute window of my walks afforded me two and a half "cycles" through the space, and once I realized what was happening, I tried to use subsequent "visits" to the place to explore elements of the landscape I hadn't encountered prior.

Notable affective experiences, annotated with stars in the map, include my discovering that I left behind circular ripples in the water. As noted above, in these digital spaces where the material qualities of space are dictated by invitations and inhibitions scripted into them by human designers, sometimes the presence of a mundane material or physical property, such as water ripples (or the movement of the sun through the sky), can be charged with both a sense of surprise and of significance. Sacramento also elicited memorable experiences that were likely less intended by the designer, at points where boundaries in the space seemingly broke down. On the right side of the map are curtains of water. I was surprised to be able to walk through them, then further surprised to find the area beyond full of angelfish, indicating Forneau had anticipated my arrival. There's a certain playful back-and-forth here, in breaking a presumed rule of the place, then discovering that that "break" is accommodated by it.

This experience invited another moment of boundary-breaking, reflected in the map by my moving straight from east to west toward the setting sun. As I approached the sun, it grew larger, indicating that it was a drawn object in space which I could approach, and, in fact, as it was setting, I walked through the sun and arrived on the other side before being whisked back to the starting point. I initially read this whisking as a sort of punitive measure returning me to the territory I had escaped, but realized that it was in fact just that the sun had set and my time was over.

Reviewing the video of my walk highlights a few factors that complement the mapping. At 1:20, my gaze is drawn to the sky by flying birds, and at 2:20 I find myself walking backwards and looking down to see the ripples I leave in the water. This both point to ways incidental details in virtual environments can impact the direction of our gaze, and invite unusual ways of moving through them. In a traditional "game" space, I likely wouldn't be invited to slowly walk backwards while looking at my "feet," but Sacramento elicited this behavior from me. There are also a few moments where surprise clearly registers on my face. The first time the experience ends itself and I find myself back at the title screen, I am visible surprised, leaning back, eyebrows raised. The two major boundary transgressions (through the waterfall and through the sun) are also met with visible (and audible, if my microphone set up had been properly working) "wow"/"woah" expressions. There's also a moment at 1:45 where I notice building structures that are only visible in the periphery of my digital gaze, and I can see my moving my physical gaze to focus on them briefly, highlighting a degree of distance between these two modes of gazing in this virtual space.

References

Anable, A. (2018). Playing with feelings : Video games and affect. University of Minnesota Press.

Manning, E. (2013). Choreography as mobile architecture. Performance Paradigm (9). http://www.performanceparadigm.net/index.php/journal/article/view/134/133

Powell, K. (2016). Multimodal mapmaking: Working toward an entangled methodology of place: Multimodal mapmaking as emplaced methodology. Anthropology & Education Quarterly, 47(4), 402-420.

Richardson, T. (2014). A schizocartography of the University of Leeds: Cognitively mapping the campus. Journal of Social Theory, 23(10), 140-162.

Steam chart for every day. (2020, March 29). [Chart generated tracking all users of the gaming platform Steam.] Retrieved on March 31, 2020 from https://steamdb.info/app/753/graphs/

Truman, S. E. & Springgay, S. (2016). Propositions for walking research. In P. Burnard, E. Mackinlay, & K. A. Powell (Eds.), The Routledge international handbook of intercultural arts research (pp. 259-267). Routledge.

Verbeek, P.P. (2006). Materializing Morality – design ethics and technological mediation. Science, Technology and Human Values, 31(3), 361-380.