A3: Immersive Book Talks
Inspiration
My name is Sarah, and I hold a BA in History and English, a BEd, and a Teacher-Librarian Diploma from UBC. I currently work as a teacher-librarian in Atlantic Canada. Prior to returning to Canada I worked as a Teacher-Librarian in China and discovered Aurasma, one of the first widely used AR applications at the time (it has since been discontinued). It was here that I started experimenting with AR and books. Publishers sometimes create book trailers in order to build up interest and excitement around new books. I had always viewed books as a way to enter new words, and programmed Aurasma to project book trailers on top of the books, to metaphorically allow students a sneak peak inside the hidden world of the story. While the students were amazed and engaged with the technology, the lack of access to YouTube, and extremely slow internet ended the project before it truly began. Despite this, the idea of using AR to encourage reader interest in books never left me.
Aurasma/HP Reveal
Aurasma, later renamed HP Reveal was shut down February, 2020.
Screen_Recording_20231121_200704_Halo AR.mp4Previous Attempts
Here is a recreation of what I had previously attempted to create in China, using the Halo AR application.
Image Credit: (Reloriasd, 2013)
"Our attention is a limited resource. There are only so many waking hours in the day, and therefore only so many things we can focus on. When we pay attention to one thing, we’re not paying attention to something else. This fact of life has been deeply complicated by technology. With more information and more choices at our fingertips than ever before, there are unprecedented demands on our attention (Center for Humane Technology. 2021)."
There is a term used in library studies, called the "Home Run Book," it's that magical book that checks off many of a student's boxes and makes them realize reading can be fun. If a student ends up reading several 'home run books' they often become an avid reader. The problem is that it is rare for a child to find a 'Home Run Book' on the first try, it often takes several tries to find the right one, and with each strikeout, it becomes harder and harder to retain a student's attention economy, and encourage them to keep trying different books. If a student has not developed an interest in reading, encouraging them to invest some of their attention economy into even reading the back of a book, let alone a few chapters can be extremely challenging. Attempting to gain some students' attention, and help them develop their reading stamina can be a difficult task, especially when one has to compete with things like video games, social interactions, social media, and numerous other things to are stimulating and engaging. The attention economy has become a hot commodity.
Publishers are already trying to engage with various forms of media to help garner interest in books, one prime example is book trailers, which work exactly the same as movie trailers. Many publishers will create trailers for upcoming highly anticipated new releases. While some students may not be interested in reading the back of a novel, many of them will engage with a 30-second to one-minute video. Scholastic often releases a YouTube playlist of book trailers prior to their book fairs. Despite these attempts to incorporate new media to engage children in books, AR appears to be an underutilized resource, with vast future possibilities.
Image Credit: (Reallusionblog, 2018)
Immersive Book Talks
Combining AR with books is not a new concept. Many different publishing companies have already combined AR and published texts in various ways:
For Further Reading See: Augmented Reality in Books and Publications
From having scenes pop off the pages of graphic novels, to providing links to additional
resources, to project 3D models onto textbook diagrams, there have been many iterations of AR used in the book industry (Kotobee, 2022).
Mock-up Example:
Image Credit: (Download Premium Png of Black Cellphone Screen Template Transparent Png by Benjamas About Transparent Hand Holding Phone, Iphone, Phone Png, Hand Phone, and Phone 2023071, n.d.; Glittering Golden Stream of Sparkles. Abstract Vector Illustration - Canvas Prints - Myloview, n.d.; Harry Potter and the Chamber of Secrets - Transparent Dobby the Elf, HD Png Download(500x950) - PinPng, n.d.; & Cockburn Library, n.d.)
Despite this, there appears to be little in the way of using AR for book talks. I see a possible future where using AR technology, whether it is a mobile device or smart glasses, students will be able to see a projection of a 3D character from a novel in their library or classroom, who will provide a book talk about why the student should read the book. This could include a general synopsis of the story, or a dramatic recreation of a scene with one or more characters, and/or if you liked books A, B, or C, then you would definitely like my book. As AI applications continue to advance, it may even be possible in the future for students to ask these 3D characters a limited number of questions about the books.
Image Credit: (i.materialise, 2017)
The Technology Already Exists
I believe that AR book talks will be a reality in the future, because the technology to create them already exists, though not within one application. In order to create this envisioned AR book talk, several different types of applications and various skill sets would need to be employed. The first would be to create or obtain a 3D model of a character from the novel. There are numerous websites where one can download 3D models for free. Since many book characters are copyrighted, 3D models of them are often classified as fanart, and in most cases cannot be sold, and can therefore be downloaded free for personal use. However, if a 3D model of a character could not be obtained, one could be created using various free 3D modeling software like SketchUp, Tinkercad or Blender.
Rigging
Though a 3D model is stationary, and while it can be projected using AR, there would be no engagement with it beyond that point, unless the 3D model has been rigged. Rigging is the process of adding an internal skeletal frame to a 3D model. This allows the computer to understand which parts of the 3D model can bend, and in what ways. These rigged joints can then be programmed to move, animating the 3D model.
More information about rigging can be found, but watching the following video, which demonstrates how a 3D model is rigged (this is a demonstration, not a tutorial video), or by reading the following article by Adobe, Rigging and skeletal animation: what it is and how it works.
Motion Capture
Once a 3D model is rigged, one could program the animation process through the computer, but it was through my research through this assignment that I learned about applications like Radical and Rokoko, which utilized mobile technology to create motion capture. Applications like these can analyze videos created on mobile phones using AI technology, and generate an animation based on the video that can be applied to 3D models.
Though it should be noted that these types of technology only capture, the body's movements, and not facial expressions. Animating facial expressions in order to synchronize with an audio file would require motion capture that specializes in facial expressions like Adobe Character Animator, or VRoid Studio.
Possible, But Not Practical
As one can see, using AR to have a 3D character present a book talk is possible with today's technology, though it is not practical at this time. To create one, an individual would need to have a variety of different skills related to video animation and 3D modelling, have knowledge and access to a variety of different applications, and have an extensive amount of time to create. It is not until all of these various forms of technology can be condensed down into one application, or the process becomes more user-friendly that would notion be realistically possible. Though I do believe within the decade I will see 3D characters giving book talks in my library, whether it is through files provided by publishers, or through collaborative cross-curricular projects created by students the potential is there, and I'm excited to greet it.
Image Credits
Aurasma | Auganix.org. (n.d.). Auganix.org. https://www.auganix.org/hud/aurasma/
Cockburn Library. (n.d.). New curved shelves in children’s library @ Coolbellup Library. Flickr. Retrieved November 26, 2023, from https://www.flickr.com/photos/cockburnlibraries/5056838352/
Download premium png of Black cellphone screen template transparent png by Benjamas about transparent hand holding phone, iphone, phone png, hand phone, and phone 2023071. (n.d.). Rawpixel. https://www.rawpixel.com/image/2023071/hand-with-phone-mockup-png
Glittering golden stream of sparkles. abstract vector illustration - Canvas prints - myloview. (n.d.). Myloview. https://myloview.com/canvas-print-glittering-golden-stream-of-sparkles-abstract-vector-illustration-no-B55288E
Harry Potter And The Chamber Of Secrets - Transparent Dobby The Elf, HD Png Download(500x950) - PinPng. (n.d.). Pinpng.com. https://www.pinpng.com/picture/JhxxwJ_harry-potter-and-the-chamber-of-secrets-transparent/
i.materialise. (2017, January 6). How to Get Started with 3D Modeling: An Interview with 3D Modeling Expert Jonathan Williamson. 3D Printing Blog | I.Materialise. Retrieved November 26, 2023, from https://i.materialise.com/blog/en/how-to-get-started-with-3d-modeling-an-interview-with-3d-modeling-expert-jonathan-williamson/
Reallusionblog. (2018, November 15). AR POST: Creating 3D characters and animations for augmented reality children’s book. Reallusion Magazine. https://magazine.reallusion.com/2018/11/15/ar-post-creating-3d-characters-and-animations-for-augmented-reality-childrens-book/
Reloriasd. (2013, August 17). Keeping video game distractions to a minimum with these helpful tips. Serious Video Game Advice for All to Follow; WordPress. https://freevideogamesplay.wordpress.com/2013/08/17/keeping-video-game-distractions-to-a-minimum-with-these-helpful-tips/
References
Center for Humane Technology. (2021, August 17). The Attention Economy: Why do tech companies fight for our attention?. Center for Humane Technology. https://www.humanetech.com/youth/the-attention-economy
Kotobee. (2022, June 29). Augmented Reality in Books and Publications - Kotobee Blog. Kotobee Blog. Retrieved November 26, 2023, from https://blog.kotobee.com/augmented-reality-in-books/