Description

An individual’s experience of the world and of others is deeply rooted in their body. Almost a century ago Merleau-Ponty (1945) already stated “I do not have a body, but I am my body”. Considering the body as “our anchor in the world”, he considered the individual as the body itself, “a perceiving subject: the point of view of the world, the time-space structure of the perceiving experience” (Martins 1992).

However, studies in human interactions have historically been logocentric, focusing mainly on verbal productions and ignoring most embodied phenomena “viewed as background noise or performance” (Bucholtz & Hall, 2016). They tend to “conceptualize the body as secondary to language rather than as the sine qua non of language” (Ibid.). This logocentric tradition disregards the fact that “bodies and embodiment are central to the production, perception, and social interpretation of language” (Ibid.).

Indeed, social interactions are to be regarded as body-to-body interactions by nature (Cosnier, 2004). It gradually became “obvious that gestures and facial expressions play a fundamental role in utterance building and in the regulation of interactions” (Cosnier, 2005). This growing interest in the embodied aspect of social interactions, was inspired especially “by the work of Goffman (1959, 1968) on the symbolic and visible significance of the body in public space, in social arenas, and in the interactional order [...].” (Mondada, 2019).

Arguing against the dichotomy between language and context, Goodwin stated that “the construction of action through talk within situated interaction is accomplished through the temporally unfolding juxtaposition of quite different kinds of semiotic resources” (Goodwin, 2000). In opposition to the monomodal theory of communication – focusing on verbal utterances – the multimodal approach highlights the ways in which individuals make use of various modes of communication in co- constructing their (inter)actions. Kress and van Leeuwen (2001) define the concept of “mode” as any semiotic resource that produces meaning in a social context (Rippl, 2015). They argue that, be it either face-to-face or distance, synchronous or asynchronous, every instance of communication relies on more than one mode to make meaning (Ibid.). Communicative modes include verbal utterances as well as gestures, facial expressions, gazes, body postures, movements, spatial arrangements, embodied manipulations of objects, etc. All communication is, and has always been, multimodal (Kress and van Leeuwen, 1996). Even more so when involving digital medias which open to a wide range of audiovisual resources (images, videos, gifs, emojis, hyperlinks, hashtags, etc.) introducing plurisemioticity in social interactions (Paveau, 2013). Furthermore, other nonlinear elements are to be taken into account when analyzing interactions: individuals often engage simultaneously in multiple activities while interacting with one another (Haddington et al. 2014). Mondada (2019) thus points out that “in multiactivity, the issue of how different trajectories of action are temporally coexisting, intertwining, and mutually shaping each other or not casts crucial light on how multiple temporalities are finely coordinated”. Finally, multisensoriality needs to be fully considered as well in interaction analysis insofar as “one contemporary challenge consists in fully recognizing that participants have multiple ways to sensorially engage in the material world, using multimodal resources not only to communicate or make their interactions accountable but also to express, manifest, and display their sensory access to the world.” (Mondada, 2019).

A growing body of research in Human Sciences re-examine human behavior through the lens of embodiment and artifaction. We observe indeed a shift “from word-and-sentence-based anthropological thought to image-and-sequence-based anthropological thought” (Pink, 2007). More specifically, visual ethnography “logically proceeds from the belief that culture is manifested through visible symbols embedded in gestures, ceremonies, rituals, and artifacts situated in constructed and natural environments” (Ibid.). Consequently “if one can see culture, then researchers should be able to employ audiovisual technologies to record it as data amenable to analysis and presentation” (Ibid.). Visual ethnography (Ruby 1996 ; Banks et Morphy 1997 ; Pink 2007) finds its roots in Bateson and Mead’s fieldwork (1942) involving visual data not as mere proof but as real empirical materials in their own right (Dion, 2007).

Indeed, “video is not simply a data collecting tool but a technology that participates in the negotiation of social relationships and a medium through which ethnographic knowledge is produced” (Pink, 2007). Audiovisual materials are not to be regarded as simple research objects but rather considered as indispensable elements of a methodological situated practice (Chauvin & Rex, 2015). The audiovisual recording of a specific social group is made under the authority of a researcher who has their own freedom over practices of shooting, assembling, editing, analyzing and presenting data (Ibanez-Bueno, 2019). However, this freedom involves negotiations with the recorded participants, their informed consent and most of all, an acute skill set. Thus, “although metaphors of ‘‘recording’’, ‘‘capturing’’, ‘‘acquiring’’, and ‘‘gathering’’ are often used to speak about the constitution of video corpora in the social sciences, video data are neither ‘‘offered’’, ‘‘found’’, nor ‘‘given’’. Rather, they are actively assembled by a range of practices” (Mondada, 2009). Those practices include for instance choosing the right equipment, the adequate locations for the camera and audio recorders, the relevant time frame and then digitizing in particular formats, assembling data in specific ways, selecting and editing shots. Finally, researchers can turn the visual data into a final publishing material guiding the reader-spectator through semiotic and narrative enrichment realized in post-production. Video thus constitutes a mode of analytical representation in itself that follows a scenario created beforehand by the researcher. Video capsules as dynamic illustrations form an innovative method of restituting data analysis and participate in the renewal of multimodal interaction analysis by making use of the technological tools available to researchers in Digital Humanities.