Perceptual Objects in Synthetic Realities
Computing power has provided us with artifacts that create new perceptual settings for objects, situations and events. The objects in question are interactive and are conveyed with various display technologies like the LCD-screens of GameBoys, desktop computer screens, VR-headsets or with overlay-technologies that augment the perception of the ordinary world.
I defend a version of realism for these kinds of objects, which is the view that we have real perceptions of them, and that they for the most part are veridical. A realistic account of perception involves answering the following three questions: 1. What are the objects in question? 2. How does causal interaction with them determine the contents of the perceptual states? 3. What is the character of the intentional consciousness that the perceptual states have towards their contents?
I hold that what we perceive in these environments are the carriers of representational roles, such as the patches of light on a computer screen, or images projected on the retina. However, to explain their role in perceptual content, we need an account of their relationship with perceptual capacities, their representational roles, and their objecthood, which is what I aim to do in this presentation.
The images on GameBoy or the volumetric projections of Star Wars start out as artifacts that serve communicational functions. We can say the consciousness in these cases involve having the represented objects, events or states of affairs as intentional objects. A common story for games is that perceptual experience in these cases involves fictional make-believe, imagination or seeing as/in (cf, Walton 1991; Wollheim 1980) and the intentional objects will not be the contents of what is properly called perceptions, given that proper perception involves a degree of epistemic success.
Here are two considerations that indicate that the intentional objects cannot be what is merely fictionally imagined for the type of interactive environments that we find in VR, games and augmented reality. First, it seems clear that the objects can be the same whether they occupy a minor part of everyday perception, or whether they are globally encompassing, as in the skeptical brain-in-a-vat scenarios. But these types of perceptions can be global, which would not be possible if they were fictional. Second, consider what is happening when a user is interacting with an avatar in a computer game. There is an important difference due to whether or not the user is interacting with the image. If you perceive your character walking without your interaction, it seems correct to hold the intentional object is a fictional walking. However, if the avatar is under your intentional control, then it seems correct to say that something more than an imagined walking occurs as the intentional object. The element of responsibility for the outcome of actions seems to have consequences for the reality status of the objects being perceived, which should lead us to look for a realist account that makes the perceptions out to be true (Sageng 2012).
We find one such realist account in David Chalmers’ “virtual digitalism” (Chalmers 2018). According to this view, objects in questions are data structures that generate sensory impressions functionally similar to those found in ordinary perception, and the perceptual content is to be given with reference to these data structures. I hold that there are three problems with this position (Sageng 2018): First, data structures are not ideally suited for satisfying the causal role, since the machinational operations they instantiate are multiple realized, and hence only serve to identify tokens rather than types. Secondly, such data structures are neither necessary nor sufficient, since it is possible for the objects to exist without them, using means alternative to digital computation. Third, these objects do not successfully individuate the perceptual objects in question. E.g. when playing Rocket League, each game session will constitute a different ball, despite being generated by the same data structure.
My proposal is that the relationship between perception and its object in these environments has a rather different story. Overall, what is happening when images take over traditional perceptual roles, is that technology via images is used to hijack the biologically instilled capacities to track objects and apply basic phenomenal categories in a perceptual field. The crucial element here is the fact that the individual is performing actions with the objects that are encountered in perceptual space, and this causes a reference shift from the intentional object conveyed by the representational role, to the former images themselves. (Sageng 2012) The fact that they occur as perceptual objects is due to the fact that the basic phenomenal schemata, such as space, time, objecthood and modal features are “virtualized”, which means that we should understand the resulting realty as a synthetic reality, rather than a virtual reality, since the resulting objects are replacements or extensions, rather than virtual counterparts of objects in the ordinary world (Sageng 2019).
This proposal has the following consequences for the central features of perception: As to their nature of its objects, they are to be regarded as artifactually sustained but self-standing substances that build on existing perceptual schemata, but due to the nomic emancipation grounded in computing power are free to add new elements of identity, modality and persistence conditions. The causal types that serve to identify the contents are determined by the artifact intentions and by the identification practices in question, which means that the determination is radically judgment-dependent. So, the epitome of synthetic reality is found in computer games rather than brain-in-a-vat scenarios. With regard to their role in intentionality, we can say that most actual cases are impure, they involve knowledge and awareness of fictional roles and its relationship to the image. Being artifactually phenomenalistic objects, their identity conditions are tied to how they are suited to aggregate knowledge and perform epistemic projections in their environments, as is the case for ordinary substances as well. (Millikan 2000).
Aarseth, Espen. ‘Doors and Perception: Fiction vs. Simulation in Games’. Intermédialités: Histoire et Théorie Des Arts, Des Lettres et Des Techniques, 1 January 2007
Chalmers, David J. ‘The Virtual and the Real’. Disputatio 9, no. 46 (2017): 309–52.
Sageng, John R.. ‘In-Game Action’. In The Philosophy of Computer Games, edited by John R. Sageng, Hallvard Fossheim, and Tarjei Mandt Larsen, 219–32. Philosophy of Engineering and Technology 7. Springer Netherlands, 2012.
Sageng, John R. “Synthetic Reality”, position paper presented at seminar with David Chalmers at ITU, 2018.
Ruth Garrett Millikan. On Clear and Confused Ideas: An Essay about Substance Concepts. Cambridge University Press, 2000.
Walton, Kendall L. Mimesis as Make Believe: On the Foundations of the Representational Arts. New Ed edition. Cambridge, Mass.: Harvard University Press, 1993.
Wollheim, Richard 1980 “Seeing-as, Seeing-in, and Pictorial Representation”, in Art and Its Objects, 2nd edition, Cambridge, MA.: Cambridge University Press, 205-26.