Ai-Driven interactive storytelling

Problem Formulation

In interactive storytelling, the audience is an active participant of a story, able to perform actions that shape the story’s progression (e.g., in a video game). There are several advantages of interactive storytelling. First, interactive stories allow the participants to feel that their actions within the narrative world are meaningfully influencing the story, increasing their sense of agency and engagement. Second, when interactive stories are tailored to their audience, they can have a higher appeal and relevance; in entertainment settings, this can lead to more enjoyment. Third, more narrative variety and a higher replay value can be achieved. Finally, interactive stories are well suited for training environments, where a story can be modified dynamically to emphasize necessary pedagogical points and make the training more effective by scaffolding the learner’s problem solving.

Although this process of audience-adaptive, interactive storytelling has existed for thousands of years, its computational simulation has only recently begun to be explored; modern personal computers, game consoles and mobile devices provide the first widely accessible technological medium in which both the presentation of a story (through video and sound) and the capture of audience feedback (in the form of keystrokes, finger swipes, Kinect gestures, etc.) can be automated and interleaved. The key research problem of interactive storytelling is how to balance the need for a coherent story with audience agency. In response, over the last twenty years the field has developed a number of experience managers. An experience manager is an AI agent that observes an audience’s actions and decides how to respond to them by modifying the narrative world. The modifications need to be consistent with the story setting as well as the events that transpired earlier. Additionally, authorial goals need to be satisfied (e.g., the wolf must eat the grandma in The Little Red Riding Hood tale). Accordingly, most existing work on experience management in interactive storytelling draws heavily from AI decision-making technology such as automated planning, search and machine learning.

Our Contributions

We have made progress towards creating an AI interactive storyteller. First, we developed PaSSAGE (Player-Specific Stories via Automatically Generated Events), one of the first experience managers to combine both player modeling and decision-making in an integrated approach. In PaSSAGE, story adaptation is achieved through two interleaved processes: player preference learning and story decision-making. To evaluate PaSSAGE we have developed two complete video games using commercial video-game engines. Both games were medieval role-playing fantasies, and the player’s preferences were encoded as inclinations to five styles of play (fighter, method actor, storyteller, tactician, power gamer) borrowed from Laws’ guide for pen-and-paper role-playing games. When the player performs an action, PaSSAGE updates its model of the player, which is maintained as a vector of scalars (one for each style of play). PaSSAGE uses the model to dynamically select between alternative subsequent events in a story, toward maximizing a designer-specified objective function. We have used two objective functions to date: the player’s enjoyment of the story and the player’s feeling of agency. In both cases, PaSSAGE was able to improve player experiences as empirically measured with some of the largest user studies in the field (a total of over 1,000 player-hours to date).




Second, we combined the player model used in PaSSAGE with a formal representation of stories as plans. Building on a recent experience manager, ASD (Automated Story Director), we described player actions as planning operators. An automated planner was then used to accommodate the player’s actions by re-planning the unfolding story. We made two significant extensions to ASD. First, if several alternative stories are computed by the planner to accommodate the player’s action, the one most likely to optimize the objective function is selected. We used PaSSAGE’s player model to estimate the objective function value for each of the candidate stories. Second, ASD pre-computed all possible ways in which the player can break the initial story through their actions. For each such action, ASD then pre-computed a contingency story off-line, before the game even began. We sped up the planning process sufficiently for an on-line operation, allowing the new experience manager to react to the player’s actions on-line, informed by the player model. The resulting system, called PAST (Player-specific Automated STorytelling), was empirically evaluated via a large-scale user study. In the process, we also empirically evaluated ASD for the first time. The user study demonstrated positive effects of planning and player modeling on player fun and agency in an interactive storytelling setting.

Third, we developed a novel theoretical framework, called GEM (Generalized Experience Management), that unifies a wide range of existing academic and commercial work in experience management and interactive storytelling. The framework represents the human player as an agent operating within a Markov Decision Process (MDP). The player takes actions according to his/her policy and collects rewards (e.g., enjoyment). Unknown to the player, an AI experience manager observes the player and changes the MDP on the fly by manipulating its transition function. In doing so, the manager attempts to increase the cumulative reward to be collected by the player over his/her experience in the game. The transition function manipulation is subject to constraints specified by the author (e.g., world settings or desired events to happen). GEM brings several benefits to the field of narrative experience management. First, it allows for more direct comparisons of diverse existing systems. Second, it enables picking and matching components of existing systems, now described in the same terms, to build a new one. Third, it opens a possibility of using the extensive apparatus of MDP and Reinforcement Learning to reason about player experiences and automated experience management.

Fourth, we developed the first story-based approach for live color commentary in sports. The approach was based on machine-learning a two-stage mapping from game states to appropriate stories that would constitute timely color commentary. We implemented the approach in the Sports Commentary Recommendation System (SCoReS) and evaluated it via user studies and interviews with professional color commentators.

Work in Progress

We are currently working on adding an appraisal-based emotion model to PAST which will allow its experience manager to accommodate the player's actions specifically to keep the player on an author-specified emotional trajectory. Our testbed for this project is iGiselle --- an interactive version of the Romantic ballet Giselle.

Publications

  1. Vadim Bulitko and David Thue. A Call for Flow Modeling in Interactive Storytelling. Advances in Cognitive Systems, Cognitive Systems Foundation, volume 4, 25–34. 2016.
  2. Sergio Poo Hernandez and Vadim Bulitko and Marcia Spetch. Keeping the Player on an Emotional Trajectory in Interactive Storytelling. In Proceedings of the Eleventh AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE). 2015.
  3. Vadim Bulitko and Greg Lee and Sergio Poo Hernandez and Alejandro Ramirez and David Thue. Techniques for AI-driven Experience Management in Interactive Narratives. In S. Rabin (Ed.) Game AI Pro. Volume II, pages 523 - 533. 2015.
  4. Alejandro Ramirez and Vadim Bulitko. Automated Planning and Player Modelling for Interactive Storytelling. IEEE Transactions on Computational Intelligence and AI in Games. 2014.
  5. Alejandro Ramirez and Vadim Bulitko. Player-specific Automated Storytelling. In the Playable Experience track at the Tenth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE). 2014.
  6. Sergio Poo Hernandez and Vadim Bulitko and Emilie St.Hilaire. Emotion-based Interactive Storytelling with Artificial Intelligence. In Proceedings of the Tenth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE). 2014.
  7. Sarah Beck and Vadim Bulitko and Sergio Poo Hernandez and Emilie St.Hilaire and Nora Stovel and Laura Sydora. Women with Wings: The Romantic Ballerina Then and Now. Abstract in Proceedings of Grace Hopper Conference. Phoenix, Arizona. 2014.
  8. Mark Riedl and Vadim Bulitko. Interactive Narrative: An Intelligent Systems Approach. Artificial Intelligence magazine. Volume 34, number 1. pages 67 - 77. 2013.
  9. David Thue and Vadim Bulitko and Howard Hamilton. Implementation Cost and Efficiency for AI Experience Managers. In the Proceedings of the Intelligent Narrative Technologies (INT) workshop of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE). Boston, MA. October 2013.
  10. Sergio Poo Hernandez and Vadim Bulitko. A Call for Emotion Modeling in Interactive Storytelling. In the Proceedings of the Intelligent Narrative Technologies (INT) workshop of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE). Boston, MA. October 2013.
  11. Greg Lee and Vadim Bulitko and Elliot Ludvig. Automated Story Selection for Color Commentary in Sports. IEEE Transactions on Computational Intelligence and AI in Games. PP(99). Pages 12. doi: 10.1109/ TCIAIG.2013.2275199. 2013.
  12. Alejandro Ramirez and Vadim Bulitko and Marcia Spetch. Evaluating Planning-Based Experience Managers for Agency and Fun in Text-based Interactive Narrative. In Proceedings of the Ninth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE), pages 65-71. Boston, Massachusetts. 2013.
  13. Alejandro Ramirez. Automated Planning and Player Modelling for Interactive Storytelling. M.Sc. thesis. Department of Computing Science. University of Alberta. Edmonton, Alberta. 2013.
  14. Greg Lee and Vadim Bulitko and Elliot Ludvig. Sports Commentary Recommendation System (SCoReS): Machine Learning for Automated Narrative. In Proceedings of the Eighth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE), pages 32-37. Stanford, California, 2012.
  15. Alejandro Ramirez Sanabria and Vadim Bulitko. Telling Interactive Player-specific Stories and Planning for it : ASD + PaSSAGE = PAST. In Proceedings of the Eighth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE), pages 173-178. Stanford, California, 2012.
  16. Greg Lee. Automated Story-based Commentary for Sports. Ph.D. thesis. Department of Computing Science. University of Alberta. Edmonton, Alberta. 2012.
  17. David Thue and Vadim Bulitko. Procedural Game Adaptation: Framing Experience Management as Changing an MDP. In the Proceedings of the Intelligent Narrative Technologies (INT) workshop of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE). Boston, MA. 2012.
  18. David Thue and Vadim Bulitko and Marcia Spetch and Trevon Romaniuk. A Computational Model of Perceived Agency in Video Games. In Proceedings of the Seventh AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE), pages 91-96. Stanford, California, 2011.
  19. Mark Riedl and David Thue and Vadim Bulitko. Game AI as Storytelling. In book: Applied Research in Artificial Intelligence for Computer Games. Springer USA. Pages 125-150. 2011.
  20. David Thue and Vadim Bulitko and Marcia Spetch and Trevon Romaniuk. Player Agency and the Relevance of Decisions. In Proceedings of the Third International Conference on Interactive Digital Storytelling (ICIDS). Edinburgh, UK. Pages 210 - 215. 2010.
  21. Greg Lee and Vadim Bulitko. Automated Storytelling in Sports: A Rich Domain to be Explored. In Proceedings of the Third International Conference on Interactive Digital Storytelling (ICIDS). Edinburgh, UK. Pages 252 - 255. 2010.
  22. David Thue and Vadim Bulitko and Marcia Spetch and Michael Webb. Socially Consistent Characters in Player-Specific Stories. In Proceedings of the Artificial Intelligence and Interactive Digital Entertainment conference (AIIDE). Pages 198 - 203. 2010.
  23. David Thue and Vadim Bulitko and Marcia Spetch and Michael Webb. Exaggerated Claims for Interactive Stories. In Proceedings of the International Conference on Interactive Digital Storytelling (ICIDS). Guimarães, Portugal. Pages 179-184. 2009.
  24. David Thue and Vadim Bulitko and Marcia Spetch. Making Stories Player-Specific: Delayed Authoring in Interactive Storytelling. In Proceedings of the First Joint International Conference on Interactive Digital Storytelling (ICIDS), pages 230 - 241. Erfurt, Germany. 2008.
  25. David Thue and Vadim Bulitko and Marcia Spetch. PaSSAGE: A Demonstration of Player Modelling in Interactive Storytelling. In Proceedings of the Fourth Conference on Artificial Intelligence and Interactive Digital Entertainment. AAAI Press. Stanford, California, USA, pages 226 - 227. 2008.
  26. David Thue and Vadim Bulitko and Marcia Spetch. Simulating the Adaptive Behaviour of Storytellers in Computer Video Games. In Proceedings of the Tenth International Conference on the Simulation of Adaptive Behavior (SAB), Last Minute Results track. Osaka, Japan. 2008.
  27. David Thue and Vadim Bulitko and Marcia Spetch. Player Modelling for Interactive Storytelling: A Practical Approach. In S. Rabin (Ed.) AI Game Programming Wisdom, Charles River Media, Inc.: volume 4, pages 633 - 646. 2008.
  28. David Thue and Vadim Bulitko and Marcia Spetch and Eric Wasylishen. Interactive Storytelling: A Player Modelling Approach. In Proceedings of the Artificial Intelligence and Interactive Digital Entertainment conference (AIIDE), pages 43 - 48, Stanford, California. 2007.
  29. David Thue and Vadim Bulitko and Marcia Spetch and Eric Wasylishen. Learning Player Preferences to Inform Delayed Authoring. In Proceedings of the AAAI Fall Symposium on Intelligent Narrative Technologies, volume Volume FS-07-05, pages 158-161. Arlington, Virginia. 2007.
  30. David Thue. Player-informed Interactive Storytelling. M.Sc. thesis. Department of Computing Science. University of Alberta. Edmonton, Alberta. 2007.

Presentations

  1. Interactive Storytelling, Artificial Intelligence and iGiselle. Invited talk at Arts-based Research Studio, Department of Secondary Education. University of Alberta. Edmonton, Alberta. February 5, 2015.
  2. iGiselle: Modeling Player’s Emotions for Interactive Storytelling. Interdisciplinary Colloquium. University of Alberta, Edmonton, Alberta. October 24, 2014.
  3. From Human Writers to AI Experience Managers. Liquid Narrative Group. North Carolina State University. Raleigh, North Carolina. October 8, 2014.
  4. PAST: Player-Specific Automated STorytelling. A playable experience spotlight presentation at the AAAI conference on AI and Interactive Digital Entertainment (AIIDE). Raleigh, North Carolina. October 6, 2014.
  5. Emotion-based Interactive Storytelling with AI. A poster spotlight presentation at the AAAI conference on AI and Interactive Digital Entertainment (AIIDE). Raleigh, North Carolina. October 6, 2014.
  6. Managing Interactive Experience with Artificial Intelligence. Invited talk for Computational Media and Design program. University of Calgary. Alberta. September 29, 2014.
  7. Interactive Storytelling for Fun and Training. Invited talk at CogSem, Department of Psychology. University of Alberta. Edmonton, Alberta. September 12, 2014.
  8. AI-based Interactive Experience Management. Invited Keynote at Replaying Japan conference. August 23, 2014.
  9. AI-based Interactive Experience Management. GRAND workshop invited talk. University of Alberta. April 7, 2014.
  10. Experience Management with Artificial Intelligence. DiscoverE Open House. University of Alberta. November 30, 2013.
  11. iGiselle. University of Alberta Interactives. November 8, 2013.
  12. Experience Management with Artificial Intelligence. Lockheed Martin. October 18, 2013.
  13. Experience Management with Artificial Intelligence. MIT Media Lab. October 15, 2013.
  14. Learning Player Preferences for Better Interactive Stories. University of Alberta International. May 6, 2013.
  15. Learning Player Preferences for Better Interactive Stories. University of Alberta Liberal Arts Day. May 3, 2013.
  16. Learning Player Preferences for Better Interactive Stories. University of Alberta Interactives. April 3, 2013.
  17. Automated Story-based Commentary for Sports. AIIDE Conference. Stanford, California. October 10, 2012.
  18. Building and Evaluating an AI Game Master. Thompson Rivers University. February 2, 2012.
  19. Building and Evaluating an AI Game Master. University of British Columbia Okanagan. December 2, 2011.
  20. Building and Evaluating an AI Game Master. Pure Speculation ’11 conference invited talk. November 19, 2011.
  21. Building and Evaluating an AI Game Master. University of Alberta. November 18, 2011.
  22. Agency for Everyone: A New Focus for the PaSSAGE Project. UC Santa Cruz. October 14, 2010.
  23. PaSSAGE: Past, Present, and the Road Ahead. Google, Mountain View, California. August 3, 2009.
  24. PaSSAGE: Past, Present, and the Road Ahead. Disney Imagineering, Glendale, California. July 31, 2009.
  25. PaSSAGE: Past, Present, and the Road Ahead. Reykjavik University. November 15, 2008.
  26. PaSSAGE:Past, Present, and the Road Ahead. Crowd Control Productions. Reykjavik, Iceland. November 12, 2008.