The Human Factor in AI Red Teaming:
Perspectives from Social and Collaborative Computing
CSCW 1-day hybrid workshop San José, Costa Rica November 10th, 2024
Embedded Files
ABSTRACT Rapid progress in general-purpose AI has sparked significant interest in "red teaming," a practice of adversarial testing originating in military and cybersecurity applications. AI red teaming raises many questions about the human factor, such as how red teamers are selected, biases and blindspots in how tests are conducted, and harmful content's psychological effects on red teamers. A growing body of HCI and CSCW literature examines related practices—including data labeling, content moderation, and algorithmic auditing. However, few, if any, have investigated red teaming itself. This workshop seeks to consider the conceptual and empirical challenges associated with this practice, often rendered opaque by non-disclosure agreements. Future studies may explore topics ranging from fairness to mental health and other areas of potential harm. We aim to facilitate a community of researchers and practitioners who can begin to meet these challenges with creativity, innovation, and thoughtful reflection.
Questions? Contact Alice Qian Zhang: aqzhang [at] andrew.cmu.edu
WORKSHOP LOGISTICS Applications for participation are now extended to September 15th A.O.E. We encourage in-person participants to apply at this time! Full conference registration is NOT required to participate in the workshop. Participants may also attend online or in person. See the conference website for more details on how to register and costs.
WORKSHOP DETAILS We welcome 20-30 academics and practitioners who are working, researching, or interested in red teaming from fields including but not limited to CSCW, AI, HCI, sociology, communications, philosophy, psychology, and labor studies. Workshop participants will experience an interactive red teaming exercise and a panel discussion among the senior authors. Afterwards, the participants and organizers will spend the afternoon together to propose future research and design agendas. We anticipate publishing a synthesis of the workshop's findings, as well as building a lasting AI red teaming research network. We will focus on the following themes:
Conceptualization of Red Teaming: Inspired by Robert Soden and colleagues’ argument to ground CSCW in history, we aim to understand the trajectory of red teaming as a socio-technical, collaborative practice. This theme invites participants to engage in deeper discussions about red teaming complexities and consider the impact of conducting research in this space. What constitutes red teaming, and how has its conceptualization evolved over time? What role does red teaming play within broader frameworks of Responsible AI, and how can decentralized or external approaches contribute to its effectiveness?
Labor of Red Teaming: This theme explores a human aspect of AI red teaming, investigating stakeholders involved in the practice and their impact on shaping AI systems to inform future practices and policies. By examining the labor arrangements and power dynamics involved in red teaming practices (e.g., inequities in organizational practices of tech labor), we seek to uncover historical parallels and contemporary methodologies that illuminate red teamers’ roles and operational frameworks. What can historical precedents teach us about red teaming as a labor practice? How can we employ diverse methodologies to investigate red teamers’ labor structures, including recruitment procedures and institutional commitments?
Well-being of and Harms Against Red Teamers: Building on the theme of labor, this theme focuses on the safety and well-being of red teamers. We will identify strategies and interventions to mitigate potential harms from exposure to harmful content during red teaming activities. By addressing these critical concerns and integrating recommendations to prioritize worker well-being, we aim to foster a culture of well-being within the AI red teaming community. How can organizations build safeguards and design interventions to protect red teamers from potential harm? How can these strategies be implemented to ensure the safety and well-being of red teamers in their roles?
ORGANIZING TEAM We are an interdisciplinary team of researchers with academic and industry experience in responsible AI, labor, moderation, mental health and well-being.
Alice Qian Zhang is a PhD student in the Human-Computer Interaction Institute at Carnegie Mellon University. Her research is dedicated to ensuring the safety of individuals engaging with social computing technologies and AI systems, with a particular emphasis on underrepresented populations and the implications for mental health and well-being.
Ryland Shaw is a pre-doctoral research assistant at Microsoft Research’s Social Media Collective, where he works on questions about AI ethics, sociotechnical norms and imaginaries, and corporate tech responsibility. He has an MA in Communication from Simon Fraser University and comes from a background in documentary filmmaking.
Jacy Reese Anthis is the director of the Sentience Institute, a visiting scholar at the Stanford Institute for HumanCentered Artificial Intelligence (HAI), and a PhD candidate in the sociology and statistics departments at the University of Chicago. Jacy researches human-AI interaction and machine learning, particularly the rise of digital minds and how humanity can work together with highly capable AI systems.
Ashlee Milton is a PhD candidate in computer science at the University of Minnesota, focusing on human-computer interaction. Their research investigates how information retrieval systems are used by and affect users from marginalized populations from a user perspective to better design these systems to support users’ needs and mental well-being.
Emily Tseng is a postdoctoral researcher at Microsoft Research. Her work explores how computing technologies mediate individual, interpersonal, and structural harms, and how to create more equitable tech. Emily publishes at top-tier venues in HCI and design (CHI, CSCW), computer security and privacy (USENIX Security), and medicine (JAMA). She earned a Ph.D. in Information Science at Cornell University and a B.A. at Princeton University.
Jina Suh is a Principal Researcher in the Human Understanding and Empathy group at Microsoft Research. Her work lies at the intersection of technology and human well-being, where she examines the role of technologies, design choices, development practices, and values embedded in them in shifting power dynamics and affecting individual and organizational mental health and well-being. She received her Ph.D. in Computer Science at the University of Washington.
Lama Ahmad is a Policy Researcher at OpenAI, leading red teaming and researcher access efforts. Her work focuses on evaluating the socio-technical impact of AI systems on society. Prior to OpenAI, Lama was at Facebook, assessing the impact of social media on elections and democracy.
Ram Shankar Siva Kumar founded and leads the AI Red Team at Microsoft and co-authored Not with a Bug, But with a Sticker: Attacks on Machine Learning Systems and What To Do About Them. He is also a Tech Policy Fellow at UC Berkeley, wherein his work on adversarial machine learning appeared notably in the National Security Commission on Artificial Intelligence (NSCAI) Final report presented to the United States Congress and the President.
Julian Posada is an Assistant Professor of American Studies at Yale University and a member of the Yale Law School’s Information Society Project and the Yale Institute for Foundations of Data Science. Their research integrates theories and methods from information science, sociology, and human-computer interaction to examine how technology is developed and used within various historical, cultural, and social contexts.
Benjamin Shestakofsky is an Assistant Professor of Sociology at the University of Pennsylvania. His research centers on the relationship between work, technology, organizations, and political economy. He is the author of Behind the Startup: How Venture Capital Shapes Work, Innovation, and Inequality.
Sarah T. Roberts is an associate professor at UCLA specializing in Internet and social media policy, infrastructure, politics and culture, and the intersection of media, technology, and society. She is the faculty director of the UCLA Center for Critical Internet Inquiry (C2i2). Informed by feminist Science and Technology Studies perspectives, Roberts is keenly interested in the way power, geopolitics, and economics play out on/via the internet, reproducing, reifying, and exacerbating global inequities and social injustice.
Mary L. Gray is a Senior Principal Researcher at Microsoft Research, a Faculty Associate at Harvard University’s Berkman Klein Center for Internet and Society, and a MacArthur Fellow. An anthropologist and media scholar by training, she focuses on how people’s everyday uses of technologies transform labor, identity, and human rights. She maintains a faculty position in the Luddy School of Informatics, Computing, and Engineering with affiliations in Anthropology and Gender Studies at Indiana University
Page updated
Google Sites
Report abuse