The Drawing Apprentice is the first co-creative drawing partner that can collaborate with users in real time on a shared canvas. It analyzes the user's lines and generates its own responses based on the user's input and previous feedback. It uses a combination of machine learning algorithms to recognize the user's sketched object, as well as their positive and negative feedback through time. The system is meant to engage users in a creative dialogue to help inspire new ideas, creatively engage the user, and emphasize the creative process over product (lowering the barrier of entry for novices). The Drawing Apprentice serves as an experimental platform to explore the technical approaches and interaction designs that help facilitate co-creation. It helps answers questions about what it means to collaborate with a creative computer and how people ideally imagine co-creation with a computer.
Developing a new cognitive framework, called creative sense-making, to understand interaction dynamics of open-ended improvisational collaboration (e.g. engagement dynamics, rhythm of interaction, nature of turn taking, leading/following strategies). The framework includes interaction analysis software for analyzing video data in a way that helps quantify different types of (participatory) sense-making in creative experiences between humans and between humans and co-creative agents.
Creative Sense-Making Cognitive Framework
Creative sense-making (CSM) is a cognitive framework to help understand the dynamic creative process through the lens of interaction and participatory interaction among many agents. It uses the cognitive science theory of enaction and participatory sense-making to describe how agents gradually interact with each other and their environment to build meaning through negotiating shared experiences in dynamic coordination. CSM helps operationalize the ideas of emergence as they relate to social relationship and dynamic interactions.
Computational Play Project
This project studied human collaboration in the domain of pretend object play in order to inform the design of a co-creative system. We found humans dynamically build improvisational meaning in the moment that grows through interaction. To support this type of real time meaning construction, we developed a co-creative play agent that can learn how to improvise on actions through demonstration in a virtual environment.
LuminAI: Co-creative Dance Environment
Helped project leads Mikhail Jacob and Duri Long analyze and evaluate the user experience and interaction design of the co-creative dance agent and LuminAI dome exhibit. This project enables users to dance with virtual agents that respond to the user's dance moves in real time. This project explores interactive machine learning and performance based co-creative systems.
The first workshop on co-creation will be hosted in Atlanta, Georgia at the International Conference on Computational Creativity on June 19, 2017. Co-creation is a new interdisciplinary research topic that asks questions related to human-computer interaction, computational creativity, and cognitive science. It investigates how two or more agents interact to produce emergent meaning and artifacts in a participatory process of meaning building (i.e. participatory sense-making). We invite researchers and creative practitioners involved in research projects related to co-creation to participate. The workshop is designed to discuss and share technical, theoretical, and empirical findings related to the unique challenges of co-creation as an application domain.