Technical Projects
For this project, I designed and implemented the interaction design and system design on my own using a mixed-method approach, including developing personas, user journeys, paper prototypes, and mockups. The Drawing Partner is a co-creative drawing agent that analyzes the user’s contribution, feedback, and instructions to produce a creative contribution on a shared canvas. The system is loosely turn-based, but the user can draw at the same time as the agent, which makes synchronous collaboration possible. The system is also a quantified co-creative AI system that quantifies and models co-creation based on the creative sense-making framework. The AI Drawing Partner is a publicly available web app (https://drawing-partner.vercel.app).
I co-taught the Georgia Institute of Technology's Technical Arts Practicum (TAP) with Dr. Brian Magerko during 2016-2017. In this role, I advised the concept development, design, implementation , evaluation, and research dissemination of computational interactive artworks. In the class, we collaborated with local artists from the community of Atlanta to authentically engage students in the artistic creative process. The works from this dual semester course were presented at a local art gallery called EyeDrum, which was open to the broader Atlanta community. The feedback and response from the students, school, and community were positive.
In this project, I was the user experience lead and project lead. The Drawing Apprentice is an early co-creative drawing partner that can collaborate with users in real time on a shared canvas. It analyzes the user's lines and generates its own responses based on the user's input and previous feedback. It uses a combination of machine learning algorithms to recognize the user's sketched object, as well as their positive and negative feedback through time. The system is meant to engage users in a creative dialogue to help inspire new ideas, creatively engage the user, and emphasize the creative process over product (lowering the barrier of entry for novices). The Drawing Apprentice serves as an experimental platform to explore the technical approaches and interaction designs that help facilitate co-creation. It helps answers questions about what it means to collaborate with a creative computer and how people ideally imagine co-creation with a computer.
QuantCollab: Collaborative Drawing Environment to Quantify Co-Creation
QuantCollab is a drawing program that quantifies elements of a drawing collaboration performed within the program and displays the results of the analysis to the user. It is designed to help support drawing collaboration by providing information and statistics about the collaboration to the users. In this way, the collaboration can become a game where the participants try to maximize their score and statistics through interacting in the program and in so doing improve their collaboration.
In this project, I assisted with the project design, concept design, and experiment design. The Creative Sketching Partner is a co-creative sketching tool that presents visually and conceptually similar sketches as the user's input sketch. The system is meant to help designers explore more of the design space and reduce design fixation, i.e. becoming stuck on one idea. The system is designed to facilitate conceptual shifts in user's by presenting either visually similar or conceptually similar sketch results. The parameters for visual and conceptual similarity are controlled by the user. This project explored co-creative AI in the domain of design.
I assisted with the user experience and concept development of this project. This research explores a new research platform for collecting exploratory sketch data. The Sketch Master game is designed to help players learn to draw from memory. The gamified aspects relate to perception, memory, and sketching. The data collected can inform sketch research about the role of perception and memory in the sketch production process.
I assisted in the user experience research of this project. Dancing on the Desktop is a multimodal spatial computing design environment. It includes a topdown projected image and side projected image plus a depth sensor to visualize a three dimensional space that is gesturally augmented. Visual images (such as icons) and text are projected onto the user's hands to indicate action items within the system. Dancing on the Desktop enables embodied interaction in a design environment, with a gestural interaction zone in between the plan and perspective view of the architectural model simultaneously. This prototype pushes the boundaries of current CAAD systems in a low-cost manner.
In this project, I assisted in the user experience and analysis. The Tactile Teacher is a wearable technology that connects the student and teacher in a new dimension: by enabling the student to sense the rhythm the teacher is tapping. The teacher glove have sensors that indicate when a tap was present, and the student gloves have vibratory motors to indicate the tapping. The study showed that, for students with no musical ability, the tapping improved accuracy by 13%.
Helped project leads Mikhail Jacob and Duri Long analyze and evaluate the user experience and interaction design of the co-creative dance agent and LuminAI dome exhibit. This project enables users to dance with virtual agents that respond to the user's dance moves in real time. This project explores interactive machine learning and performance based co-creative systems.