Ai-Driven interactive storytelling

Problem Formulation

In interactive storytelling, the audience is an active participant of a story, able to perform actions that shape the story’s progression (e.g., in a video game). There are several advantages of interactive storytelling. First, interactive stories allow the participants to feel that their actions within the narrative world are meaningfully influencing the story, increasing their sense of agency and engagement. Second, when interactive stories are tailored to their audience, they can have a higher appeal and relevance; in entertainment settings, this can lead to more enjoyment. Third, more narrative variety and a higher replay value can be achieved. Finally, interactive stories are well suited for training environments, where a story can be modified dynamically to emphasize necessary pedagogical points and make the training more effective by scaffolding the learner’s problem solving.

Although this process of audience-adaptive, interactive storytelling has existed for thousands of years, its computational simulation has only recently begun to be explored; modern personal computers, game consoles and mobile devices provide the first widely accessible technological medium in which both the presentation of a story (through video and sound) and the capture of audience feedback (in the form of keystrokes, finger swipes, Kinect gestures, etc.) can be automated and interleaved. The key research problem of interactive storytelling is how to balance the need for a coherent story with audience agency. In response, over the last twenty years the field has developed a number of experience managers. An experience manager is an AI agent that observes an audience’s actions and decides how to respond to them by modifying the narrative world. The modifications need to be consistent with the story setting as well as the events that transpired earlier. Additionally, authorial goals need to be satisfied (e.g., the wolf must eat the grandma in The Little Red Riding Hood tale). Accordingly, most existing work on experience management in interactive storytelling draws heavily from AI decision-making technology such as automated planning, search and machine learning.

Our Contributions

We have made progress towards creating an AI interactive storyteller. First, we developed PaSSAGE (Player-Specific Stories via Automatically Generated Events), one of the first experience managers to combine both player modeling and decision-making in an integrated approach. In PaSSAGE, story adaptation is achieved through two interleaved processes: player preference learning and story decision-making. To evaluate PaSSAGE we have developed two complete video games using commercial video-game engines. Both games were medieval role-playing fantasies, and the player’s preferences were encoded as inclinations to five styles of play (fighter, method actor, storyteller, tactician, power gamer) borrowed from Laws’ guide for pen-and-paper role-playing games. When the player performs an action, PaSSAGE updates its model of the player, which is maintained as a vector of scalars (one for each style of play). PaSSAGE uses the model to dynamically select between alternative subsequent events in a story, toward maximizing a designer-specified objective function. We have used two objective functions to date: the player’s enjoyment of the story and the player’s feeling of agency. In both cases, PaSSAGE was able to improve player experiences as empirically measured with some of the largest user studies in the field (a total of over 1,000 player-hours to date).

Second, we combined the player model used in PaSSAGE with a formal representation of stories as plans. Building on a recent experience manager, ASD (Automated Story Director), we described player actions as planning operators. An automated planner was then used to accommodate the player’s actions by re-planning the unfolding story. We made two significant extensions to ASD. First, if several alternative stories are computed by the planner to accommodate the player’s action, the one most likely to optimize the objective function is selected. We used PaSSAGE’s player model to estimate the objective function value for each of the candidate stories. Second, ASD pre-computed all possible ways in which the player can break the initial story through their actions. For each such action, ASD then pre-computed a contingency story off-line, before the game even began. We sped up the planning process sufficiently for an on-line operation, allowing the new experience manager to react to the player’s actions on-line, informed by the player model. The resulting system, called PAST (Player-specific Automated STorytelling), was empirically evaluated via a large-scale user study. In the process, we also empirically evaluated ASD for the first time. The user study demonstrated positive effects of planning and player modeling on player fun and agency in an interactive storytelling setting.

Third, we developed a novel theoretical framework, called GEM (Generalized Experience Management), that unifies a wide range of existing academic and commercial work in experience management and interactive storytelling. The framework represents the human player as an agent operating within a Markov Decision Process (MDP). The player takes actions according to his/her policy and collects rewards (e.g., enjoyment). Unknown to the player, an AI experience manager observes the player and changes the MDP on the fly by manipulating its transition function. In doing so, the manager attempts to increase the cumulative reward to be collected by the player over his/her experience in the game. The transition function manipulation is subject to constraints specified by the author (e.g., world settings or desired events to happen). GEM brings several benefits to the field of narrative experience management. First, it allows for more direct comparisons of diverse existing systems. Second, it enables picking and matching components of existing systems, now described in the same terms, to build a new one. Third, it opens a possibility of using the extensive apparatus of MDP and Reinforcement Learning to reason about player experiences and automated experience management.

Fourth, we developed the first story-based approach for live color commentary in sports. The approach was based on machine-learning a two-stage mapping from game states to appropriate stories that would constitute timely color commentary. We implemented the approach in the Sports Commentary Recommendation System (SCoReS) and evaluated it via user studies and interviews with professional color commentators.