Organisers
Joost Broekens, Leiden University (NL) , contact: joost.broekens@gmail.com
Jeffrey Girard, University of Kansas, (USA)
Desmond Ong, University of Texas at Austin, (USA)
Summary
The Affective Computing Pre-Conference features state-of-the-art research on computational techniques for recognizing, modelling and shaping human emotion, and emphasizes ways to strengthen interdisciplinary connections between the computational and human sciences of emotion.
Submitted contributions and invited speakers discuss the latest research on computational methods inspired by the affective sciences, as well as how such methods can transform research in affective science. For submitted contributions, we will be accepting extended abstract submissions for Poster and Flash Talk presentations (instructions below). This year’s focus is “Opportunities, challenges, and impacts of computational methods in emotion research”.
This year, we are also organizing a special session on research using Large Language Models and affective science, across the range of basic and applied research questions, for example: how well do these models understand emotions; what can we learn from the nature/representation of emotions in these models; how can we use these models to generate artificial stimuli, or label data; how can we use these models in applied systems (e.g., for emotional well-being).
Embedding and previous conferences
This pre-conference is sponsored by the Association for the Advancement of Affective Computing (AAAC). The pre-conference is part of the affective computing preconference series started in 2017 in Boston at SAS, and the 2019 and 2022 pre-conferences at ISRE. The Affective Computing preconference attempts to support the increasing interest in the interdisciplinary study of emotion in which computational modelling as well as computational analysis play an important role.
Abstract Submissions and registration
We will be accepting abstract submissions for Poster and “Flash Talk” presentations. Presentations should address topics in affective computing, broadly defined, e.g., emotion recognition, emotion modeling, emotion expression in technological artifacts, and human (affective) responses to (affective) technology. Extended abstracts should emphasize the interdisciplinary nature of the work and explicitly link with the affective sciences. We welcome original work and work that has been published or will be published elsewhere. Acceptance of the contribution is at the discretion of the program committee based on a light review which will focus on (1) clarity and (2) relevance to both computational methods as well as emotion research. Extended abstracts should be 2-page PDFs including bibliography and should outline the goals of the research, methods used, and a summary of the results. Position and/or “blue sky idea” extended abstracts are also welcome but need to make their arguments clearly and concisely. Submissions are welcome from any career stage/status. Extended abstracts will be published online, but not through an official publisher, meaning that you keep the copyright and you are still able to submit elsewhere.
Submissions are submitted through Easychair: https://easychair.org/my/conference?conf=acisre2024
Registration for the preconference is through the ISRE main conference, see their website: https://www.isre2024.org/registration/
Deadlines
Submission deadline: Friday 19 April 2024 (final deadline!)
Acceptance notification: 10 May 2024 (a bit delayed)
Conference format
The pre-conference is an in-person physical one-day event during ISRE. It features invited talks and contributed “flash talks”, a poster booster session and a poster session. Further, time is allocated for discussion. In particular, we plan a plenary moderated discussion on the relevance of LLMs for the affective sciences, practically as well as theoretically.
List of speakers and panelists
Aleksandra Cichocka, Professor of Political Psychology, University of Kent.
Jon Gratch, Professor of Computer Science and Psychology, University of Southern California
Dirk Heylen, Professor of Human Media Interaction, University of Twente
Rachael Jack, Professor of Computational Social Cognition, University of Glasgow
Gale Lucas, Assistant Professor of Computer Science and Civil and Environmental Engineering, University of Southern California
Aleksandra Cichocka, Professor of Political Psychology, University of Kent.
Politicians’ use of national identity rhetoric on social media predicts engagement and electoral success
Social media platforms are playing an increasingly important role in politics. Many recent political movements are thought to have gained traction through their leaders' savvy use of social media to promote appealing narratives about the nation and its identity (e.g., ‘Make America Great Again’). Yet, the effectiveness of such strategies remains unclear. We examined online and offline success of US and UK politicians’ social media rhetoric expressing different national sentiments. We compared expressions of a positive national identity, highlighting national pride and attachment to one's country, to expressions of a defensive national identity, portraying the nation as exceptional and entitled. We validated the use of a large language model (GPT-3.5) to code for the presence of the different types of national identity rhetoric in posts to X (nSt1 = 234,906; nSt2 = 226,900; nSt3 = 296,416). Defensive rhetoric received more likes and more reposts when used by right-wing politicians, but it did not benefit left-wing politicians online and predicted a smaller share of their votes in a congressional election. Expressing a positive identity was linked to more likes and greater vote share for politicians on both sides. This research establishes a link between the use of identity rhetoric and online attention, as well as electoral success. The findings can contribute to debates about the role of identity appeals amplified via social media in contemporary politics.
Gale Lucas, Assistant Professor of Computer Science and Civil and Environmental Engineering, University of Southern California
Are LLMs Effective Negotiators? Systematic Evaluation of the Multifaceted Capabilities of LLMs in Negotiations
Negotiation is a core task for studying emotional feelings and expressions in human decision-making. Being a mixed-motive task, it creates both interpersonal and intrapersonal conflicts for the negotiators. Motivational tensions often arise when negotiations pit aspirations for individual accomplishments against the demands of sustaining social connections. This leaves difficult decisions for the negotiators about working towards their own outcomes or making sacrifices for others. Such situations can be fraught with emotional encounters. For instance, as a negotiator strives to get as much as possible for themselves, in doing so, they need their partner to go along as well. A negotiator that tries to take too much can annoy their partner, and in turn, hurt their likeability in the eyes of their partners and also the partner’s affective evaluation of the outcome. Instead, it is desirable for the negotiator to strive for maximum performance while ensuring that the partner is satisfied and leaves with a positive perception of the partner. Using three different approaches from Affective Computing, we discuss results that illustrate ways to address the challenge of considering both motives when AI negotiate with humans. First, by conducting human-computer interaction research using the paradigm of Wizard of Oz, we show that 1) humans can be trained by AI to improve their ability to get better outcomes for themselves in a negotiation, and 2) humans have more positive emotional responses to practicing negotiation with AI than with other humans. Second, by adapting machine learning models (specifically using reinforcement learning or RL), we show that, while traditional approaches to RL overemphasize getting good outcomes for the AI (at the cost of actually coming to agreement with the human negotiation partner), updating these models using knowledge from economics/psychology allows AI to more successfully strike a balance between meeting their own needs and making the human partner happy (enough to accept the deal). Finally, by turning to the burgeoning work in from Natural Language Processing on Large Language models (LLMs), we are able to demonstrate that, above other language models, GPT has the best performance and understanding of objective outcomes such as one’s own points, but it is worse at estimating important subjective outcomes relevant to emotions, such the partner’s affective evaluation of the outcome and of the AI itself.
Rachael Jack, Professor of Computational Social Cognition, University of Glasgow.
Equipping Artificial Agents with Facial Expression Intelligence Using Psychological Science
Artificial agents are now increasingly part of human society, destined for schools, hospitals, and homes to perform a variety of tasks. To engage their human users, artificial agents must be equipped with essential social skills such as facial expression communication. However, many artificial agents remain limited in this ability because they are typically equipped with a narrow set of prototypical Western-centric facial expressions of emotion that lack naturalistic dynamics (Jack 2013) . Our aim is to address this challenge by equipping artificial agents with a broader repertoire of socially relevant and culturally sensitive facial expressions (e.g., complex emotions, conversational messages, social and personality traits). To this aim, we use new, data-driven and psychology-based methodologies that can reverse-engineer dynamic facial expressions using human cultural perception (Yu, Garrod et al., 2012) . We show that our human-user-centered approach can reverse engineer many different, highly recognizable, and human-like dynamic facial expressions that typically outperform the facial expressions of existing artificial agents (Chen, Garrod et al., 2018, Chen et al., 2019) . By objectively analyzing these dynamic facial expression models, we can also identify specific latent syntactical signalling structures that can inform the design of generative models for culture-specific and universal social face signalling (Jack, et al., 2016) . Together, our results demonstrate the utility of an interdisciplinary approach that applies data-driven, psychology-based methods to inform the social signalling generation capabilities of artificial agents (Jack and Schyns 2017) . We anticipate that these methods will broaden the usability and global marketability of artificial agents and highlight the key role that psychology must continue to play in their design.
Program (Room: Bann: PFC/0G/024)
09:15 - 10:15 Welcome / Keynote
AC@ISRE Introduction (Desmond Ong/Joost Broekens)
Keynote: Aleksandra Cichocka: Identifying national sentiments in language
10:15 - 10:45 Coffee break / Poster Session
10:45 - 12:00 Session on Large Language Models (LLMs) (Chair: Desmond Ong)
Keynote: Gale Lucas: Are LLMs Effective Negotiators?
Guy Laban, Tomer Laban and Hatice Gunes: LEXI: Large Language Models Experimentation Interface [link]
Yasaman Etesam, Özge Nilay Yalçın, Chuxuan Zhang and Angelica Lim: Can Fine-Tuning Vision Language Models Unlock Contextual Emotion Recognition? [link]
Kristina Schaaff and Marc-André Heidelmann: Impacts of Anthropomorphizing Large Language Models in Learning Environments [link]
12:00 - 12:30 Discussion session on LLMs as method for affective computing (Moderator: Desmond Ong)
12:30 - 13:30 Lunch Break
13:30 - 14:45 Session on Emotion Theory and Computation (Chair: Gale Lucas)
Keynote: Jon Gratch: On the Use of GPTs in Computational Modelling of Emotion Theory.
Alessandra Yu, Daniela Schiller and Leroy Lowe: The Human Affectome: A Teleological Framework for Organizing Algorithms Across Human Affective Phenomena [link]
Bernhard Hilpert, Kim Baraka and Joost Broekens: The disambiguating function of affect: an integrated perspective. [link]
Elena Walsh: Does a face speak for itself? Automated facial expression analysis and 'explainable’ AI [link]
14:45 - 15:15 Coffee break / Poster Session
15:15 - 16:10 Session on affective robots and agents (Chair: Jon Gratch)
Keynote: Rachael Jack: Equipping Artificial Agents with Facial Expression Intelligence Using Psychological Science
Joanna Komoszyńska, Dominika Kunc, Bartosz Perz, Adam Hebko, Przemysław Kazienko, Maciej Behnke and Stanislaw Saganowski: Designing and Executing a Large-scale Real-life Affective Study [link]
16:10 - 17:00 Plenary discussion (with expert panel) on the interplay between computational methods and affective science (Moderation: Desmond/Joost)
Expert panel: Jon Gratch, Gale Lucas, Alexandra Cichocka, Rachael Jack, Dirk Heylen
18:15 Bus departing from the QUB campus to the Northern Ireland Assembly
19:00 - 22:00 ISRE Welcome Reception