Inquiry #12
What Happened Where and When?
Direct coding of video enables large amounts of video data to be sorted by segmenting video into clips, then coding, clustering, and comparing video clips so that actions within strips of activity can be analyzed without the intermediary step of transcribing the action into print.
An action-focused sequence can:
• display screenshot photo sequences that proceed in comic strip fashion with speech, action, or modes marked with arrows flowing across the panes, e.g., Norris (2004).
• slice up time by mediated action rather than turns of talk; put action first field by annotating individual clips in a film sequence (e.g., VideoAnt).
• enable video analysis (e.g., Transana, Studiocode) that syncs and displays multiple camera views and enables coding of video segments on a timeline to track different modes and aspects of experience.
• use hypertext tools or multimedia tools (e.g., PowerPoint or screen capture) to display unfolding activity with tool manipulation (Nelson et al., 2008). For example, see Smith and Dalton’s (2016) example of analysis of a director’s or composer’s cut that makes available the action and decision-making that created student films.
Action Transcripts
Identify a short one-minute segment of video and try out video analysis software to create an action transcript. In some video annotation programs, you’ll be able to enter a URL into the site and generate a screen that will allow you to attach annotation notes to segments of video clips. Using video analysis software, tag strips of time sequences, that is, create a typical video analysis format for the clips in the video, segmenting the video file into mini-clips separated by timestamps, which allow you to create, color-code, and cluster mini-clips of activity for closer analysis.
1. For this small timeline, track back the events that led up to this moment, using tables or diagrams to show the modal relationships among the people, materials, actions, and meanings. On the first pass through the data, label clips with each participant’s name; then use the coding tools to find groups that typically form around particular materials or locations.
2. Add layers of multimodal analysis by systematically focusing on specific modes, looking at a different mode in each pass through the data. (For example, one pass might look across instances to code the proximity of actors to one another or to particular tools; another pass might code instances where actors are gazing at the same object, signaling a shared or collaborative gaze.)
3. Video analysis programs allow you to look across an instance of activity to see what is happening in a particular instance. You are chunking your video data into little slices that are small video clips. This allows you to analyze each clip for all the possible codes in your coding scheme. This also enables close comparisons of mediated actions. In Transana for example, you can produce a visualization of the audio track and a timeline of the modes used in a clip. Hovering over a color reveals the code and the modes that were being used at different points in the timeline. A timeline view of modes allows you to look across coded instances and compare how often particular modes are used and where they co-occur.
For more details on creating and analyzing action transcripts, go to Coding Action Walkthrough.
Nelson, M. E., Hull, G. A., & Roche-Smith, J. (2008). Taking, and mistaking, the show on the road: Challenges of multimedia self-presentation. Written Communication, 25(4), 415–440.
Norris, S. (2004). Analyzing multimodal interaction: A methodological framework. London: Routledge.
Smith, B. E., & Dalton, B. (2016). “Seeing it from a different light”: Adolescents’ video reflections about their multimodal compositions. Journal of Adolescent & Adult Literacy, 59(6), 719–729.