Ture Schwebs
Bergen University College, Faculty of Education, Landåssvingen 15, N-5096 Bergen, Norway
Published: 12 March 2014
©2014 T. Schwebs. This is an Open Access article distributed under the terms of the Creative Commons Attribution-Noncommercial 3.0 Unported License (http://creativecommons.org/licenses/by-nc/3.0/), permitting all non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Citation: Nordic Journal of ChildLit Aesthetics, Vol. 5, 2014 http://dx.doi.org/10.3402/blft.v5.24169
An app is a software application typically designed to run on mobile media platforms like smartphones and tablet computers. Being equipped with touchscreens, these media utilize predefined multi-touch finger gestures like tapping, pressing and scrolling. ‘Digital’ derives from the Latin word for finger, and as Mika Elo (2012) points out, modern technology is digital even in the etymological sense of the word. Moreover, Cecilie Lindhé (2010) argues that the bodily interaction of the user with the touchscreen conveys an aesthetic of tactility: «A visual sense is born in the fingertips». Torben Grodal (2003, 139) discusses the «embodied experience» of computer games, and highlights the significance of the fact that the player is able to move within and in response to the images and sounds. In traditional media, the story progression is controlled by the author, illustrator or director. Video games require a precise hand-eye coordination and motor skills of the player, Grodal points out.
Apps were originally available for services like email and news, but this soon expanded into other categories. A major reason for this development was the launching of Apple’s iPad in 2010. In a surprisingly short time, smartphones and tablets have become significant entertainment channels, opening up a variety of genres for a varied audience. Not least, the development and marketing of apps designed for children has been extensive, and publishers and media companies offer a a wide variety of products including games, picture stories and educational material. The supply is largest in the English language, although many of the products are multi-lingual. In Scandinavia there is also a growing interest in this market, and Norwegian publishing houses launched around fifty apps for children in 2010–2012 (Prytz 2013).
The mass media have given these apps considerable attention. To a large extent, they have addressed the technological features and capabilities of the platforms, their attitudes ranging from fascination to skepticism. Most of the write-ups are consumer-oriented journalism. Critical reviews including discussions of the aesthetic aspects of apps are rare, Nina Goga (2013) claims. However, a significant contribution is Hadassah Stichnothe’s article «Engineering stories? A narratological approach to children’s book apps» (2014), which establishes a theoretical framework exemplified by analyses.
Tablet apps introduce a wide range of communication forms and telling modes. In the following discussion, the question addressed is what sort of aesthetic affordances such apps may accommodate, exemplified with a reading of an app entitledThe Fantastic Flying Books of Mr. Morris Lessmore.
The word ‘aesthetics’ was first applied in 1750 by Alexander Baumgarten in his treatise Aesthetica, to describe the sensuous experience of the fine arts. In present art philosophy, aesthetics is still related to sensuous experience. In his essay «Ästhetik und Hermeneutik» [Aesthetics and Hermeneutics] Hans-Georg Gadamer (2001) explains aesthetics as a sensuous experience of the way something is expressed, and he stresses that this sensuous experience may carry a «surplus of meaning». Gadamer also links aesthetic experiences to interpretation and the process of meaning making. Thus, the use of the concept aesthetics may accentuate not only the sensuous experience associated with modes of expression, but also the ability to perceive and understand various cultural manifestations. I will relate my exploration of the app’s aesthetics affordances to Gadamer’s understanding.
The term ‘affordance’ was coined by James Gibson (1977), with reference to a quality of a particular object or an environment that allows an individual to perform an action. For instance, on a door, a crash bar affords pushing, while a door handle affords twisting. Affordances are relative in that they are dependent on the capabilities of the acting subjects. Standard door knobs do not afford door opening for a crawling infant. An affordance is also dependent on the historic and cultural context in which it occurs.
The term is also applied in theories of social semiotics, as in Gunther Kress and Theo van Leeuwen’s (2001) examination of modes of communication. They understand these modes to be socially and culturally shaped resources for making meaning, and maintain that for any given communication purpose, some modes afford a higher degree of semiotic potential than others. For example sheet music is more suitable than words to convey how to play music, and an animated figure may capture a time lapse better than a static image.
Just as there are affordances of modes, so are there affordances of media (Seip Tønnessen 2010). Media aesthetics researchers highlight the importance of examining the medium itself in order to understand how objects appear and are perceived. The study of media aesthetics implies investigating how the meaning-making process is influenced by the material and technological1 aspects of the media (Hausken 2009, 9).
The most prevalent digital media affordances are the broad range of interactive elements, functions that are not found in conventional media. Interactivity is an ambiguous term, the meaning of which varies in different fields. In media theory, it is often used to describe various sorts of interplay between a reader (or «user») and a computer program. The central point is that the program has a «responsiveness» (Rafeli 1988) that gives the reader a feeling of meaningful «dialogue» and of being able to influence or control the reading.2
According to Gadamer, a work of art is not to be considered as a constant object. Aesthetic experiences are situated; they take place in a given time and space. Gadamer claims that a work of art is an «implementation» or cooperation involving the work and the viewer, and this recreation is part of the aesthetic experience (Bale 2009, 25). This view corresponds with the function of the user dialogues found in app stories on tablets and smartphones. The interactivities influence the reading of the story, and such actions are frequently prerequisites for the narration itself.3 The interactive elements, dissolving or disturbing the spatio-temporal relation as they do, are parts of the media-specific semiotic repository.
There are a great variety of apps on the market offering fictional stories for children. Some of them are simply reproductions of printed picturebooks. The story is copied from the book to the screen page by page, and the only «enrichment» may be an option to have the text read aloud. These apps are what I have termed ‘digital immigrants’, playing on Marc Prensky’s (2001) dichotomy.4 Correspondingly, at the other end of the scale, there are apps I will call digital natives. These texts, «born digital», are ambitious productions, exploiting the media’s visual and auditory potential, including animations, puzzles and interactive tasks. Within this category, there are all sorts of genres, including computer games, edutainment, educational games, classic children’s books adaptations and advanced literary products.
One of the first interactive book apps was Alice for the iPad (2010). The story, an adaptation of Lewis Carroll’s classic Alice in Wonderland, is 52 pages in length and contains 20 animated scenes.5 The illustrations are based on John Tenniel’s 1865 drawings. Alice for the iPad demonstrates the rich technological potential of the app media: «Shake the iPad and make the Mad Hatter madder, slosh around the Pool of Tears, play croquet with a flamingo, or tilt the iPad to make Alice grow and shrink, like you have a ‘Drink This’ game command at your disposal.» (Fitzgerald 2010).
However, the interactive digital telling of children stories goes back in history more than twenty years. In 1992, Random House publishing and Brøderbund software released a CD-ROM called Just Grandma and Me, the first in a series of pictorial stories intended for 3–7 years old.6 The series was called Living books and marketed as bringing «a whole new dimension to storytelling». Each story consisted of about a dozen tableaux depicting central scenes in the narrative, and these were brought «to life» by small animations initiated by clicking on objects. A short verbal text accompanied each tableaux, including a read-aloud option.
The Living books became popular, encouraging the production of a large number of similar texts during the following decade. Walt Disney transformed many of the company’s cartoons into «animated storybooks» under the slogan «A Story Waiting For You To Make It Happen».
Apart from the gyroscope and the finger-controlled user interface, most of the Living books features are still found in today’s apps for children. The touchscreen allows the reader to trigger animations, manipulate objects and give life to people and animals. This part of the interactivity is a two-step process: identifying where in the picture the hot spots are hidden; and discovering what they hide, and how the objects behave. These activities play on the elements of surprise and suspense associated with veiling and unveiling. In this way digital technology exploits the general fascination of the classic hide-and-seek game: searching is as much fun as finding.7
The story entitled The Fantastic Flying Books of Mr. Morris Lessmore was written by the American author, illustrator, and filmmaker William Joyce. It is available as an animated film, an iPad app and a picturebook, all produced by the Louisiana-based visual-effects company Moonbot Studios.8 The primary focus for this article is on the aesthetic aspects of the story as it appears on the app platform, but reference will be made to the film and book versions to put the app in perspective.
The central character in The Fantastic Flying Books of Mr. Morris Lessmore is Morris Lessmore, a book lover who lives in New Orleans. He is sitting on his balcony writing his memoirs when a hurricane suddenly strikes. The wind literally blows Morris’ writing out of his book, knocks him off the balcony and eventually devastates the city. Walking into the ravaged countryside, Morris becomes aware of a woman drifting overhead, holding on to a collection of airborne books. One of the books takes Morris to a magical library filled with flying books. Morris becomes the proprietor of the library, and for years he takes care of the books and lends them to people visiting the library. As an old man, he finally completes his memoirs. Miraculously, he becomes young again, and flies away, carried by the flying books.
Along with the interactive elements, to be discussed below, visual and audio features constitute an essential element of the aesthetic affordances of apps. In digital media, as in movies and in contemporary children’s literature – not least picturebooks – there is a trend to incorporate open or covert references to other literary texts, pictures and popular cultural products. Being multimodal, the opportunities for intertextual reference in narrative apps are virtually unlimited (within copyright limitations).
The Lessmore app carries a rich network of implicit and explicit allusions, citations, pastiches and references to factual events and fictional works. It is noteworthy that many of the intertextual links are to classic feature films and the movie world in general.
An obvious real-life backdrop for the story is Hurricane Katrina, which destroyed New Orleans in 2005. The protagonist (whose name is an allusion: «Less is More») is visually modeled on the silent film actor Buster Keaton, easily recognizable thanks to the «pork pie» hat. The hurricane mirrors the storm scene in Keaton’s silent movie Steamboat Bill Jr. (1928), as well as the tornado in the fantasy adventure film The Wizard of Oz (1939).
An eye-catching visual motif in the app is the woman who is being carried through the air by flying books. This motif echoes similar scenes found in the French fantasy short film Le Ballon Rouge, the children’s books character Mary Poppins and the British dancing film The Red Shoes (see Figures 1–4).
Figure 1. The Fantastic Flying Books of Mr. Morris Lessmore app: Morris discovers a flying woman.
Figure 2. Le Ballon Rouge [The Red Balloon] (1956) is a 34 min feature film about a Paris school boy who gets hold of a large balloon. A gang of envious bullies destroys it and threatens him, but other balloons come to his rescue by clustering around him and carrying him away on a ride over the city.
Figure 3. Mary Poppins is the title character of a series of eight children’s books (1934–1988) by P. L. Travers. The books tell the story of a mystical nanny who is blown into a London family home, held aloft by her magical umbrella. In 1964 Disney made a musical fantasy film, starring Julie Andrews, loosely based on Travers’s stories.
Figure 4. The Red Shoes (1948), a feature film based on a fairy tale by Hans Christian Andersen, tells the story of a young ballerina who becomes the lead dancer in a ballet. The Morris app flying woman wears a white dress and red shoes, like the ballerina, and appears in a similar posture.
The interior of the magical library seems to draw upon artistic depictions of book collections such as the well-known 1850 painting Der Bücherwurm [The Book Worm] by the German romanticist painter Carl Spitzweg. The abundant book shelves are filled with Shakespeare, Dickens, Verne and other works from the classic Western (children’s) canon. The English iconized nursery rhyme «Humpty Dumpty» is a recurrent figure throughout the story.9 Humpty makes his first appearance inside the first book that urged Morris to go to the library. Humpty appears as an animated object in 14 of the scenes, thus forming a verbal, visual and interactive leitmotif.
Not unlike The Wizard of Oz, the Lessmore app employs colors as a narrative device. During the storm in the beginning, the screen turns black-and-white, the absence of colors representing hopelessness and despair. The colors reappear gradually as the story progresses. Morris himself is tinted when he enters the library, and his memoir book does the same when he starts to write again. The metaphor is clear and unambiguous: Books will color your life.
These rich references to classic film, art and literature illustrate the multimodal and intertextual potential of the app platform, a potential that the app shares with the more traditional form of film and picturebooks. The allusions to cultural products of both adults and children also serve to highlight the cross-over character of The Fantastic Flying Books of Mr. Morris Lessmore app. It offers enjoyable titbits for readers across generations.
NAVIGATION AND ANIMATION: TWO TYPES OF INTERACTIVITY
The Lessmore app presents the story as a sequence of 27 scenes or tableaux. Each scene includes options for readers that trigger sound effects, small animations or other visual movements. This structure implies that the reader is offered two types of interactive processes, which may be termed navigation and animation.
The most crucial navigation process is the finger swiping required to change the tableau, thereby making the presentation of the story proceed. The app is not self-moving. Neglect of the «page turning» function will bring the narration to a full stop. However, if the reader remains passive, he will eventually receive discreet visual hints, like the appearance of an arrow or a dog-ear. Paratext operations are also included in navigation acts; for example, the language setting, the home screen button, the adjustments for light and sound level, and the on or off buttons for the audio narrator or the written text. The paratext options are accessed via a navigation bar (see Figure 5).
Figure 5. Humpty Dumpty comes alive when the book – the flying lady’s «favorite» – is tapped. It flips around, revealing the nursery rhyme. The vertical navigation bar of the app is to the right.
The second type of interactivity is part of the fictional structure and incites animation events of various sorts. The manipulation of objects by moving them or having them «talk» or play simple games, is a media-specific feature that is often emphasized in app reviews and sales arguments. The Lessmore app is marketed as follows: «In this reinvention of digital storytelling you can repair books, tumble through a storm, learn the piano and even get ‘lost in a book’.» (morrislessmore.com).
Unlike the navigation acts, the animation events may be understood as rudimentary narratives (Schwebs and Otnes 2006, 85). Some of them expand, enrich or enhance the main story by exemplifying or visualizing particular traits of the characters or objects. For example, in the scene showing the flying lady, the books that are carrying her will flap when tapped by the reader. The voiceover states that «Morris wondered if his book could fly. But it couldn’t. It would only fall to the ground with a depressing thud.» This statement is confirmed when the reader taps the Morris figure; he throws the book into the air only to see it drop. A second example is inside the library, where the reader can make more than a dozen books whisper classic citation, such as the opening words of Dickens’ A Tale of Two Cities: «It was the best of times, it was the worst of times» or Shakespeare’s «Now is the winter of our discontent» from Richard III.
Some of the animation events integrate text, graphics and movements in such a way that they blend into a single unit of expression. In an almost surrealistic scene, Morris walks upon the top of the letters forming the following clause: «Sometimes Morris would become lost in a book and scarcely emerge for days.» When moved along, he dives into a wide open space in which countless words pass by, giving the impression that he will disappear in outer space.
Other animation events seem to be irrelevant for the narration; these are rather small incidents or gags only loosely related to the plot. For instance, on a grand piano in the library, the reader may play the nursery rhyme and singing game «Pop! Goes the Weasel» (Figure 6). A few interactivities even seem to distract from the telling of the story; for example, in a game-like design the reader is able to rearrange biscuit letters in a cereal bowl.
Figure 6. The reader as a pianist – coached by Humpty Dumpty.
One interactive animation may function as an «anti-narrative». In the tableau showing the devastated landscape after the storm – depicted in black and white – the reader has the option of finger painting the sky blue (Figure 7). This scene precedes Morris’ discovery of the flying lady, the encounter that marks the turning point in the story. By anticipating the tint metamorphosis which is to come later – the return of colors – the reader unintentionally ruins the efficacious color metaphor.
Figure 7. The reader as a finger painter.
Figure 8. The Lessmore AR Imagnotron: a book-centric augmented reality app that brings the pages «alive».
Such narrative derailings and digressions that apparently work against the plot may still add a prolific quality to the story as a whole. First, since every scene has some embedded interactive features that the reader has to find for himself, the app is constantly inviting him to play and explore and be curious. Second, this narrative «disturbance» may act as an aesthetic quality and part of the meaning producing process, thereby expressing an important literary value, a literariness. In his «Introduction to Structural Analysis of Narratives», Roland Barthes (1977, 93) distinguishes between two types narrative events: cardinal functions (or nuclei, kernels) and catalysers (or satellites).10 The nuclei constitute the narrative backbone; they are sequences that are essential for moving the plot forward. The catalysers are optional narrative units; they may accompany the nuclei and supplement the plot, but are not necessary for it. They may add mood and atmosphere to the story. For example, when the tornado in the Lessmore story picks up a house, the reader can use his fingers to spin it around, but he doesn’t have to. In relation to the nuclei, the catalysers are «expansions». In the Lessmore app, the 27 tableaux represent the kernel sequences, while the catalysers appear as animation events.
Catalysers are consecutive events, cardinal functions are both consecutive and consequential, Barthes points out: «Catalysers and nuclei are linked by a simple relation of implication: a catalyser necessarily implies the existence of a cardinal function to which it can connect, but not vice-versa.» (1977, 97–98) From the perspective of the story, a catalyser always has a discursive function: «It accelerates, delays, gives fresh impetus to the discourse.» (1977, 95) In order to read the Lessmore story and to gain access to the animation events, the reader needs to visit all the scenes (cardinal functions) in the prearranged order, but he does not have to execute the animation events (catalysers). However, by neglecting them, he misses one of the fundamental affordances of the app aesthetics. Like the mechanical and three dimensional books and CD ROM Living Books, the Lessmore app has sought to exceed the limited and static surface of the page, thus challenging the continuous progress. It aims to construct a «borderless» or unlimited reading / playing space.
Thus, the interactive ingredients in The Fantastic Flying Books of Mr. Morris Lessmore app (and in many similar fiction apps for children) generate a noteworthy literary hybrid. They include components that establish linear, dynamic processes, like a film, but also non-processual or static spaces, like an amusement park. The epic structure of the app consists of both movements and stops, and the reader may become immerse in the story and still remain on the surface of it. In this way, the spatio-temporal division of narration is constantly challenged.
MORRIS’ STORY: PART OF THE METABOOK GENRE
The most obvious reason for telling the story of Morris and his flying books seems to be to honor the book format as a medium per se and to praise the act of reading. The basic feature constituting the narrative is a Disney-like anthropomorphization of the library books. They act like humans; they eat, sleep, walk and fly; they seek – and receive – comfort, love and care. The book as a cultural artefact is strongly present throughout the entire text.
Thus, the Lessmore story may be included in the metabook genre, denoting «stories defending the print media». In recent years, a number of books have been published thematizing and vindicating books and printed material. Recent titles include the American Lane Smith’s It’s a book (2010), the French Herve Tullet’s Un Livre (English version: Press Here) (2010), the Portuguese José Jorge Letria’s Se Eu Fosse um Livro [If I Were a Book] (2011), and the Norwegian Max Estes’ Hva handler den om? [What’s It About?] (2012). In addition, there is Good Night iPad. A Parody for the Next Generation (2011) by «Ann Droyd» that plays on the 1947 American classic bedtime story Good Night Moon by Margaret Wise Brown and Clement Hurd.11 Some of these take a playful or even ironic tone in their depiction of printed books.
Unlike many of the above-mentioned works, the Lessmore story presupposes, but also conveys, a strong sentimental and romantic view of the book medium. Moreover, the consistent anthropomorphization of books facilitates a reversal in the «media-serving-man» relationship. Not only do the printed works serve Morris, but he also becomes their servant – to the extent that what he actually reads is almost irrelevant.
This book loving story, like the adaptation of the popular and enormously influential Alice’s Adventures in Wonderland (in its original version) for the app platform, seems to indicate a new marketing strategy. If the aim is to advocate for an unknown device like an app, it is a good idea to base the promotion upon something familiar, in this case the book «platform» and particularly the children’s literature genre. Perhaps the unconditional homage to the old media is a way of justifying a new one.12
ADAPTATION AND REMEDIATION: FROM FILM TO APP TO BOOK TO AR
As mentioned, the story about the bibliophile Mr. Lessmore’s custodianship of a magical library appears on three distinct media platforms. Moonbot Studios first produced the animated short film in 2011.13 In early 2012, the company released the iPad app, and later the same year the picturebook. In addition, an «augmented reality» (AR) app was designed to go with the picturebook and released in 2012. The iPad app and the picturebook are both adaptations of the film. A comparison of the various adaptations may broaden the perspective on the aesthetic affordances of each platform.
A central reference work in the field of adaptation theory is Linda Hutcheon’s A Theory of Adaptation (2012). Hutcheon describes the product of adaptation as a «transposition» of a particular work. This transcoding process will often involve a shift of media, but can also include shifts of genres, frames or contexts (2012, 7–8). According to Hutcheon, the transpositions are recreations rather than reproductions and should be regarded as autonomous works, and interpreted and valued as such.
With regard to the Lessmore story, a comparison of what could be called «the narrative motors» of the three platforms – the film, the picturebook and the app – reveals significant variations, partly due to different aesthetic affordances.
The animated film relies almost solely on pictures. There is no narrator, no voice over, no dialog and no spoken words. Nor are there any inter-titles (title cards). It is a silent movie, but not in the sense of no sound since there is some diegetic sound (sounds made by objects in the story), like the noise of wind and storm. In addition, a simple musical theme is recurrent, creating a sentimental mood and serving as a musical leitmotif. By explicitly imitating the aesthetics of the silent movies, the Lessmore film pays homage to this bygone era.
The Lessmore picturebook draws upon the interaction of pictures and words to tell the story. On each double-spread, there is a page of written text, describing what is depicted in the picture and what happens in the story. On this point the book «mimes» the app. The text-picture interplay offers an appropriate narrative redundancy, and the combined verbal and visual storytelling allows the reader time to stop and ponder. The absence of the interactive animations that are included in the app implies a dominance of cardinal narrative functions. The book form allows for a more condensed story than the film.
Graphics, layout and design are media-specific semiotic resources, and these are used deliberately in the Lessmore picturebook. Single letters and words are scattered over some of the illustrations, sometimes in a poetic, artistic or playful way. The typeface is based on a font used in American elementary textbooks in late eighteen hundreds, thereby establishing a historical allusion to learning the art of reading.
The Lessmore app emulates central features of the film but, as mentioned, it also presents the story similar to the book, both visually (in terms of graphics and illustrations) and verbally. According to Linda Hutcheon, the narrative aspect is a fundamental premise in adaptation processes, emphasizing «how adaptations allow people to tell, show or interact with stories» (2012, 22). However, in apps for minors halts in the narration may themselves be an aesthetic quality. When a child and a grown up read the Lessmore app together there may be an interplay between the two of them, not unlike that which occurs when reading picturebooks to small children. During the reading aloud process, the mediator may stop to elaborate on the story, in what Ingeborg Mjør (2013) calls «strategies of expansion». The (adult) reader will linger on pictures, point, give names to objects, and ask questions, thereby encouraging the child to participate in a dialogue. Mjør also points out the importance of making «spaces» in digitalized picturebooks by including interactive motifs. The reader expects clicking to lead to interesting reading elements. In the Lessmore app the catalysers – the interactive events – may function as strategies of expansion. Moreover, Roland Barthes (1977, 95) claims that catalysers have, to use Roman Jakobson’s term, a phatic function. They maintain the contact between the narrator and the reader.
Moonbot Studios offers one more Lessmore reading device: an augmented reality app entitled Imagnotron. Augmented reality (AR) may be defined as a live view of a physical, real-world environment through a digital devise. The idea is to make the real-world «greater» or «stronger». The process is simple: the user downloads the Lessmore AR app to an iPad or iPhone, then focuses the camera on a page in the picturebook, allowing the app to recognize the image. Viewed through the screen, the image appears to come «alive»: characters move, books fly off the page and whisper their lines aloud, and two-dimensional images become three-dimensional (Figure 8).
Precisely how the interactive elements differ from usual app activities is yet to be examined. However, since the Imagnotron app is not only a companion to the picturebook but also dependent on it, it represents a distinct media-specific affordance. The reading process necessitates the simultaneous use of two distinct technologies; the digital technology subordinates the print technology, and the extent to which the result is an adaptation may be discussed. Nevertheless, the AR device, by enabling several layers of representation, raises complex questions regarding the relationship between levels of semiotic codes. Ultimately, it challenges borders between fiction and reality.
The picturebook and the app adaptations of the Lessmore film serve as excellent examples of Jay David Bolter and Richard Grusin’s term remediation, a word implying that new media «refashion older media and […] older media refashion themselves to answer the challenges of new media» (Bolter and Grusin, 2000, 15). The two apps remediate features of the cinematic aesthetics. The picturebook, being an adaptation that follows the first app, echoes both the film and the app. The Imagnotron app figuratively as well as literally assumes the picturebook. Thus, the three versions involve shifts in media. In these processes it is possible to identify media-specific affordances: the film relies mainly on a visual mode, the book utilizes visual and verbal modes, and the two apps operate in visual, verbal and reader interactive modes.
The narrative aspect is a premise for all the versions. All of the platforms rely upon a temporal reading; the story is meant to be read sequentially, with the elements following one another in an orderly pattern. However, the tempo may differ. The process of watching a movie is normally limited to a preset and continuous time period. In contrast, the time required to read the picturebook and the two apps will vary according to the pace at which the readers discuss and explore the double-spreads and tableaux. They have the option to slow down the temporal aspect of the reading, or even bring it to a full stop; or to speed it up by skipping animations or scenes.
Regardless of the media involved, any reading of a given text may result in a variety of interpretations. The reception will never be the same under the next reading, even by the same reader. In computer games, «readings» are particularly prone to change; for instance players in online multiplayer games contribute to the fictional universe by building their avatars. In contrast, the narrative games in the Lessmore story are designed and produced prior to reading; and unlike gamers, the readers do not construct events or add characters. Nevertheless, there are variations between the three versions. The film and the book display finite or «closed» texts, while the appearance of the app may vary with different readers and readings. One reader may activate the hurricane, while another may play the word game or listen to the talking books.
The four Lessmore versions exemplify a significant trend in transmedia storytelling. Essential parts of a story «get dispersed systematically across multiple delivery channels for the purpose of creating a unified and coordinated entertainment experience» (Jenkins 2011), and each medium contributes in its own way to the storytelling.
Furthermore, it is evident that new media have changed the contexts in which the content is produced, distributed and consumed. As Linda Hutcheon points out, «Digital is more than a platform» (2012, xxi). The digital media create numerous expectations, allowing the users to become players, explorers and readers, as well as producers of their own stories. One example of this phenomena is popular fan fiction – stories about characters or settings written by fans of an original work. Not surprisingly, the Lessmore story is being remediated and remixed on social networking sites.
CONCLUSION: LESSMORE’S AESTHETIC AFFORDANCES
The multi-touchscreen is a media-specific affordance, and the apps that run on tablets and smartphones utilize this technology, presupposing finger movements like tapping, holding and panning. The Lessmore app utilizes this affordance by adding tactile processes to the continuous interplay between reading, watching and listening. The finger gestures have a twofold task: to initiate navigation on the macro level and to activate games, puzzles and animations on the micro level.
One of the most prominent features of the Lessmore app is its rich intertextuality, with a multitude of references to other cultural artifacts. Quite a few of these are dependent on tactile movements; for example, specific movements allow readers to tip library books, to listen to excerpts and to paint the sky. These «finger prints» give readers direct access to words, pictures and sounds that appeal to audio, visual and tactile senses. The tactile feature goes beyond the page turning found in printed books. The physical reader-screen interaction stimulates a sort of excitement or «liveliness» as the text comes to life in response to the fingers of the reader.
Viewed in the light of Gadamer’s insight into aesthetics, the excitement created by the Lessmore app may be regarded as evidence of the notion that art is an experience, not an artefact. Gadamer goes on to claim that the experience of art satisfies a profound human need for playing and gaming, and he declares that the community of «work» and «audience» is actually a «festival» (Gadamer 1977).
The tactility of the Lessmore app is associated with a sensuous experience of the materiality of the medium, and empowers a potential for meaning making. Finger gestures produce and display meaning, thereby enabling the user to sense and understand. In a multimodal app, several senses and modes of perception mingle and merge, and a number of sensuous spaces for experience are activated. The tactile sense releases a sort of «synesthesia», to borrow a concept from psychology. The stimulation of one sense simultaneously influences the other senses. By bringing a variety of verbal and visual forms and designs into play, including figures, shapes, colors, sounds and movements, the Lessmore story and other app stories come into existence in a multi-sensuous way, thereby revealing a complex aesthetic convergence.
Notes
1. For instance, may mobile media units have features like gyroscope (responding to physical movements of the unit), geolocation (GPS) and face recognition.
2. Some theorists distinguish between interactivity and reactivity. Interactivity means that the program will «answer» to a number of previous user actions and to the relationship between them. Reactivity means that the program responds only to one immediately previous user actions (Rafaeli 1988).
3. «Navigation is to interactivity what montage is to film.» Tom Perlmutter, the director of the National Filmboard of Canada (referred in Hutcheon 2012, xxi).
4. In 2001 learning researcher Marc Prensky coined the terms ‘digital natives’ and ‘digital immigrants’, differentiating (young) students who grew up with «computers, video games and the Internet» from the generations who were not born into the digital world.
5. The 2010 version of the Alice app had an abridged text, later versions contain the full text.
6. The story was based on the picturebook Just Grandma and Me (1983) by the American children’s book writer and illustrator Mercer Mayer. It features Little Critter, an anthropomorphic animal character.
7. Nina Goga (1997) points out that hide-and-seek features are found in mechanical books, produced for children from the last part of the eighteen hundreds. These books, called «movable pictures», which grow popular during the next century, were furnished with mobile figures and with flaps which were to open, disclosing further elements. In this way the reader – or player – would reveal hiding places and secrets. The books exploited the growing interest of technics and mechanics of its time. The fascination included the experience of handling objects which are both dead and alive, thus crossing the border between the automatic and the imitation. In the «pop-up»-books, developed later, the book itself unfolds, constructing a 3D space for the movements and the exposing.
9. «Humpty Dumpty» is one of the best known nursery rhymes in the English-speaking world. Humpty Dumpty falls down and is broken, but nobody can «put Humpty together again.» As a character and literary allusion the figure has been referred to or appeared in a large number of works of literature, music and popular culture. In Alice in Wonderland’s sequential Through the Looking-Glass (1872) Humpty discusses semantics and pragmatics with Alice.
10. Barthes’ own French terms are ’fonctions cardinales (ou noyaux)’ and ‘catalyses’.
11. An example of a book articulating fascination of the narrative power of another media – in this case film – is Brian Selznick’s The Invention of Hugo Cabret (2007). It tells about an orphan who encounters the legendary cinema pioneer Georges Méliès in 1930s Paris, thus thematizing early French film aesthestics. The book visualizes how Méliès – originally a stage magician – saw the illusionary potential of the movies. When the reader turns the pages, the illustrations embody cinematic effects like zoom, closeups and montage. Letting sequences of wordless picture spreads alter with text pages, the book appears to include clips from silent movies. According to Selznick, he created «something that is not a exactly a novel, not quite a picturebook, not really a graphic novel, or a flip book or a movie, but a combination of all these things» (Amazon). A film adaptation, Hugo (2011), was nominated for 11 Academy Awards, winning 5.
12. Taking advantage of canonized works for marketing electronic novelties is not a new idea. In 1991 the American CD ROM production company Voyager made The Complete Annotated Alice in its Expanded Books series. At the same time scholars at Brown University developed The Victorian Web, about 1,500 documents building a colletion of interconnected and annotated Victorian literary classics, demonstrating the then new hypertext principle, thus anticipating the World Wide Web (Lanestedt 1994).
13. The film (running time 15 minutes) was directed by William Joyce and Brandon Oldenburg, and created with computer animation, miniature models and traditional hand-drawn techniques. Among many awards, it received the 2012 Academy Award for the Best Animated Short.
REFERENCES
Amazon: The Invention of Hugo Cabret (www.amazon.com/dp/0439813786).
Bale, Kjersti. Estetikk. En innføring. Oslo: Pax forlag, 2009.
Barthes, Roland. «Introduction to the Structural analysis of Narrative». In Image, Music, Text. London: Fontana Press, 1977 [1966].
Bolter, Jay David and Richard Grusin. Remediation, Understanding New Media. Cambridge, Mass.: MIT Press, 2000.
Elo, Mika. «Digital finger: beyond phenomenological figures of touch». Journal of Aesthetics & Culture. Vol. 4, 2012.
Fitzgerald, Scott. «Alice for the iPad preview». Techstyles (techstyles.com.au/alice-for-the-ipad-preview/ipad-apps), 2010.
Gadamer, Hans-Georg. The Relevance of the Beautiful and Other Essays. Cambridge University Press, 1977.
Gadamer, Hans-Georg. «Estetikk og hermeneutikk». In Lægreid, Sissel and Torgeir Skorgen (eds.). Hermeneutisk lesebok. Oslo: Spartacus, 2001 [1986].
Gibson, James J. «The Theory of Affordances». In Shaw, Robert and John Bransford (eds.). Perceiving, Acting, and Knowing. Hillsdale, NJ: Lawrence Erlbaum, 1977.
Goga, Nina. «Gemmeleg og tredimensionale tekster». PLYS, København, 1997.
Goga, Nina. «Teknologi eller estetikk? Hvordan vurdere bildebokapper». Bøygen. Vol. 3, 2013.
Grodal, Torben. «Stories for Eye, Ear, and Muscles. Video Games, Media, and Embodied Experience» In Wolf, Mark J.P. and Perron, Bernard (eds.). The Video Game Theory Reader. New York, 2003, pp. 129–155.
Hausken, Liv. Medieestetikk: Studier i estetisk medieanalyse. Oslo: Scandinavian Academic Press, 2009.
Hutcheon, Linda. A Theory of Adaptation. New York: Routledge, 2012.
Jenkins, Henry. «Transmedia 202: Further reflections» Confessions of an Aka-Fan. The Official Weblog of Henry Jenkins, 2011 (henryjenkins.org/2011/08/defining_transmedia_further_re.html). PubMed Central Full Text
Kress, Gunther and Theo van Leeuwen. Multimodal Discourse. The Modes and Media of Contemporary Communication. London: Hodder Arnold, 2001.
Lanestedt, Jon. «Elektroniske bøker». In Schwebs, Ture (ed.): Skjermtekster. Oslo, Universitetsforlaget, 1994.
Lindhé, Cecilia. «’Bildseendet föds i fingertopparna’: Om en ekfras för den digitale tidsåldern». Ekfrase: Nordisk Tidsskrift for Visuell Kultur. Vol. 1, 2010.
Mjør, Ingeborg. «Frå Apan fin (1999) til Hej då, lilla apa! (2008). Bildebok og animasjonsfilm – resepsjon og medieestetikk». Nordic Journal of ChildLit Aesthetics. Vol. 4, 2013.
Prensky, Marc. «Digital Natives, Digital Immigrants». On the Horizon. MCB University Press, Vol. 9 No. 5, 2001.
Prytz, Øyvind. Litteratur i digitale omgivelser. Arts Council, Norway, 2013.
Rafaeli, Sheizaf. «Interactivity: From New Media to Communication» In R.P. Hawkins, Wieman, J.M. and og S. Pingree (eds.). The Sage Annual Reviewof Communication Research. Vol. 16. Beverly Hills: Sage Publications, 1988.
Schwebs, Ture and Hildegunn Otnes. Tekst.no: Strukturer og sjangrer i digitale medier. Oslo, Cappelen, 2006.
Stichnothe, Hadassah. «Engineering stories? A narratological approach to children’s book apps». Nordic Journal of ChildLit Aesthetics, Vol. 5, 2014.
Tønnessen, Elise Seip. Sammensatte tekster. Barns tekstpraksis. Oslo, Universitetsforlaget, 2010.
About The Author
Ture Schwebs
Bergen University College, Faculty of Education, Bergen, Norway
Norway