#AI Clone #Proxy-Mediated Interaction
#Collaborative Technologies #Future of Work
Hi, I'm Jieun.
My research introduces "Synthetic Humans," where human ideas, behaviors, and identity features are replicated, customized, and simulated in mediated environments. I have explored how emerging media technologies (e.g., virtual avatars, synthetic voices and images, and AI clones) extend human presence and capabilities in digital environments, introducing new opportunities and challenges for interpersonal collaboration. My recent work introduces Proxy-Mediated Interaction, where a personalized AI agent that mimics individual users' identity features and interaction patterns (i.e., AI self-clones) serves as a social proxy in interpersonal settings on users' behalf.
I am a Ph.D. Candidate in Information Science at Cornell University, advised by Prof. Susan R. Fussell (Committee members: Prof. Mor Naaman, Prof. Malte F. Jung). Before joining Cornell, I completed a master's degree in Interaction Science, with a focus on Behavioral Science and Human-AI Interaction, and two bachelor's degrees in Media Communication and Informatics at Sungkyunkwan University (SKKU) in Seoul. I have received the Cornell Media Studies Grant in 2023 and 2024, and interned at Microsoft Research in 2024.
RESEARCH INTEREST
My research envisions a future where people communicate, collaborate, and connect with virtual entities that emulate human characteristics, identities, and social behaviors. My previous work addresses the questions of (1) how people perceive and interact with the autonomous agents (e.g., embodied AI agents, voice assistants) that become increasingly capable and humanlike in collaborative settings, and (2) how people express and extend their identities through virtual self-presentations (e.g., virtual avatars, synthetic voices) in mediated social interactions. Through these empirical insights, my dissertation proposes a new paradigm of virtual collaboration using AI self-clones, questioning (3) how self-mimetic agents and other forms of digitally mediated presence influence collaboration, while raising new ethical and moral questions.
With my interdisciplinary expertise in Media Psychology, Behavioral Science, and User Experience Design, I have led multidisciplinary teams of researchers and engineers to design and evaluate prototypes of novel Human-AI Interaction systems. My work provides empirical foundations to collaborative technology design by studying the psychological and behavioral mechanisms underlying human engagement with emerging technologies, using lab experiments, field studies, computational analyses, surveys, and interviews. My future work will develop interaction frameworks and design guidelines that inform users, designers, and platform developers about the social and ethical risks of synthetic representations and provide meaningful questions to support collaborative technologies for the future of work.
A central strand of my work investigates how humans perceive and interact with virtual agents. Virtual agents assist human tasks by acting as instructors and collaborators across domains. Beyond functional assistance, they can embody humanlike qualities, such as empathy, personality, and social behaviors, building interpersonal trust and bonds. Building on the Computers-Are-Social-Actors (CASA) paradigm (Nass & Reeves, 1996), my earlier research has demonstrated the interpersonal dynamics in human-agent interactions and their impact on users’ affective and cognitive responses.
Should Voice Agents Be Polite in an Emergency? Investigating Effects of Speech Style and Voice Tone in Emergency Simulation
Kim, J. & Fussell, R. S. (2025)
CHI'25 Proceeding
Million Eyes on the "Robot Umps": The Case for Studying Sports in HRI Through Baseball
Kamino et al. (2025)
HRI'25 Proceeding
Empathetic Pedagogical Agent: Mitigating Harmful Effects of Negative Feedback Through Self-Disclosure
Kim, J., Lee, K., Kim, W., Jeong, N., Kim, J., & Song, H. (2024)
International Journal of Human-Computer Interaction
Urgency Builds Trust: Emotional Expression of a Voice Agent in an Emergency Simulation
Kim, J., González-Pumariega, G., Park, S., & Fussell, S.R., (2023)
CSCW'23 Poster
Agreeableness of a Virtual Agent: Effects of Reciprocity and Need for Help
Kim, J., Sah, Y. J., & Song, H. (2021)
IMCOM'21 Proceeding
"I Can Feel Your Empathic Voice": Effects of Nonverbal Vocal Cues in Voice User Interface
Kim, J., Kim, W., Nam, J., & Song, H. (2020)
CHI'20 Extended Abstract
Collaborative Voice Agent for Blind and Low-Vision Users
Phutane, M., Kim, J., & Azenkot, S.
Using a co-designing method, this project aims to build a collaborative voice assistant that can help blind and low-vision (BLV) workers access web searches, create slides, and prepare presentations. A series of semi-structured interviews was conducted for both BLV people and their sighted companions.
Remote Teamwork: Moderating Online Group Discussion Using Voice Agents
Kim, J. & Fussell, S.R.
With the rise of remote collaboration tools, workplaces have become more interdisciplinary and multicultural, enabling individuals from different backgrounds to leverage their unique strengths and perspectives. As an attempt to build an inclusive and diverse teamwork environment, this study examines the design of an agent system that can effectively moderate group communications, particularly in an online setting.
Beyond generic agents, recent self-mimetic AI models enable personalized, context-specific interactions by simulating individual users' decision-making and behavioral patterns. To identify social and ethical implications of using such agents as interpersonal proxies, I built AI clones that mimic users' contextual knowledge, personality, relational norms, and communication styles, then examined how they function as social proxies across multiple domains. My empirical research proposes design guidelines for aligning user intent with proxy behavior and demonstrates impacts on interpersonal relationships, trust, and collaborative outcomes.
What Should My Proxy Say When It Does Not Know What to Say? Self-Disclosure and Abstention Strategy of AI Self-Clone Proxy in AI-Mediated Collaboration
Kim, J. & Fussell, R. S. (2026)
Reframing Wizard-of-Oz for Proxy-Mediated Interaction
Kim, J. & Kim, H. (2026)
CHI'26 Extended Abstract
Ice-breaking Through My AI Proxy: Effects of Social Exchange with an Unfamiliar Teammate’s LLM-based Proxy Agent on Interpersonal Collaboration
Kim, J. & Fussell, R. S. (2025)
Ditto: Embodied LLM-based Self-Mimetic Agent that Attends Meeting on my Behalf.
Internship project at MSR (2025)
Advances in media technologies allow users to transcend physical constraints and construct their idealized self-representation, creating a new form of interaction with extended selves. Users construct virtual identities by adjusting their visual and vocal representations in online environments. This selective process of self-presentation has a significant impact on how people perceive themselves and interact with others in mediated environments.
Best Version of My Voice: Strategic Voice Customization for Self-Presentation Across Online Speech Interactions
Kim, J., Tong, Y., & Fussell, S.R. (2026)
My Voice as a Daily Reminder: Self-Voice Alarm for Daily Goal Achievement
[Best Paper Honorable Mention]
Kim, J. & Song, H. (2024)
CHI'24 Proceeding
Using a Gender-Swapped Avatar in a VR Job Interview
Kim, J., Sandhaus, H., & Fussell, S.R., (2023)
CSCW'23 Poster
Effectiveness of VR Crane Training with Head-Mounted Display: Double Mediation of Presence and Perceived Usefulness
Song, H., Kim, T., Kim, J., Ahn, D., & Kang, Y. (2021)
Automation in Construction
Online social behaviors are shaped by how people use social media and interact in digital communities, influencing information sharing, emotional well-being, and interpersonal dynamics. My prior work has investigated the role of message features in spreading misinformation, methods for detecting stressors through online expression, and approaches to identifying and mitigating harassment in community platforms. These studies reveal how digital environments mediate both individual experiences and broader social outcomes.
What Message Features Influence the Intention to Share Misinformation about COVID-19 on Social Media? The Role of Efficacy and Novelty
Song, H., So, J., Shim, M., Kim, J., Kim, A., Lee, K. (2023)
Computers in Human Behavior
Effect of audio content on simple and repetitive tasks requiring eyesight: Focus on task performance and emotional responses
Nam, J., Kim, E., Kim, J., Kim, J., Lee, D., Chung, W. (2023)
Information Development
Experiencing stress during COVID-19: A computational analysis of stressors and emotional responses to stress
Han, J., Song, H., Kang, J., Kim, J., & Kim, T. (2022)
Cyberpsych, Behavior, and Social Networking
Detecting Online Harassment on Reddit Comments
Gilbert, S., Kim, J., & Matias, J.N.
Online harassment remains prevalent despite the community's efforts. In this study, the moderating intervention on online comments on Reddit was tested. From the r/technology subreddit, 9000+ comments were collected and annotated to be classified as either abusive language, bigotry, or personal attack.