Date and time: Monday 5 September 11.00 – 12.00 hours
Abstract:
Over the last several years, end-to-end neural conversational agents have vastly improved in their ability to carry a chit-chat conversation with humans. However, these models are often trained on large datasets from the internet, and as a result, may learn undesirable behaviors from this data, such as toxic or otherwise harmful language. In this talk, I will discuss the problem landscape for safety for E2E convAI, including recent and related work. I will highlight tensions between values, potential positive impact, and potential harms, and describe a possible path for moving forward.
Bio:
Emily Dinan is a Research Engineer at Facebook AI Research in New York. Her research interests include conversational AI, natural language processing, and fairness and responsibility in these fields. Recently she has focused on methods for preventing conversational agents from reproducing biased, toxic, or otherwise harmful language. Prior to joining FAIR, she received her master's degree in Mathematics from the University of Washington.
Date and time: Wednesday 2 November 11.00 – 12.00 hours
Abstract: It is crucial to develop tools for automated hate speech and abuse detection on social media platforms. These tools should help to stop the bullies and the haters and provide a safer environment for individuals, especially from marginalized groups, to express themselves. However, recent research shows that machine learning models are biased, and might make the right decisions for the wrong reasons. There is evidence that hate speech and abuse detection models associate hateful content with marginalized groups, which leads to blocking them and flagging their content on social media platforms instead of providing them with a safe environment. In this talk, I will share our work on understanding the performance of hate speech and abuse detection models and the different biases that could influence them. We demonstrate that the good performance of some NLP models is due to syntactical biases learned during pre-training. We also demonstrate that social bias is not the only type of bias in NLP models. Finally, we investigate the impact of social biases in NLP models on the performance and fairness of hate speech detection models.
Bio: I just started my 4th year as a PhD student at the University of The West of Scotland. My research interests are in social science computing and ethics in NLP. In particular, I focus in my research on hate speech detection, social bias, and fairness in natural language processing (NLP). I’m currently an enrichment student at the Alan Turing Institute in London. This summer, I did an internship at IBM Research in New York. I’m also the organizer of the women_in_NLP talk series. For more information, check my website https://efatmae.github.io/
Date and time: Wednesday 9 November 11.00 – 12.00 hours
Abstract: TBC
Bio: https://www.turing.ac.uk/people/researchers/yi-ling-chung
When: 7th Dec 11am
Where: old Robotarium Seminar room
Title: Assessing the Effects and Risks of Large Language Models in AI-Mediated Communication
Short abstract: Large language models like GPT-3 are increasingly becoming part of human communication. Through writing suggestions, grammatical assistance, or machine translation, the models help people to communicate more efficiently. Yet, we have a limited understanding of how integrating them into communication will change culture and society. For example, a language model that preferably generates a particular view may change people's minds when embedded into widely used applications. My research empirically demonstrates that embedding large language models into human communication involves systemic societal risk. In a series of experiments, I show that humans cannot detect language produced by GPT-3, that using large language models in self-presentation may damage interpersonal trust, and that interactions with opinionated models change users' attitudes. I introduce the concept of AI-mediated communication—where AI technologies like large language models augment, optimize, or generate what people say—to theorize how embedding large language models in our communication presents a paradigm shift from previous forms of computer-mediated communication.
Bio: Maurice Jakesch is a Ph.D. candidate at Cornell's Information Science Department. Advised by Mor Naaman and Michael Macy, he works on assessing the risks and effects of technologies that change how people communicate. Before Cornell, he completed a BS in Electrical Engineering at ETH Zurich and an MS in Information Technology at the Hong Kong University of Science & Technology. He also received an MA in Philosophy of Science and Technology and an honors degree in Technology Management at TU Munich. He has interned at Microsoft Research, Facebook Core Data Science, and GE's Digital Foundries.
When: 14th Dec 11am
Abstract:
To proactively offer social media users a safe online experience, there is a need for systems that can detect harmful posts and promptly alert platform moderators. In order to guarantee the enforcement of a consistent policy, moderators are provided with detailed guidelines. In contrast, most state-of-the-art models learn what abuse is from labelled examples and as a result base their predictions on spurious cues, such as the presence of group identifiers, which can be unreliable. In this work we introduce the concept of policy-aware abuse detection, abandoning the unrealistic expectation that systems can reliably learn which phenomena constitute abuse from inspecting the data alone. We propose a machine-friendly representation of the policy that moderators wish to enforce, by breaking it down into a collection of intents and slots. We collect and annotate a dataset of 3,535 English posts with such slots, and show how architectures for intent classification and slot filling can be used for abuse detection, while providing a rationale for model decisions.
When: 11 Jan 11 am
Title: Towards Multilingual Vision-and-Language Models
Abstract: There has been an explosive growth of vision-and-language architectures in the last few years. While these models have reached impressive performance on several tasks, they are usually trained on English captions paired with images from North America or Western Europe. In this talk, I will discuss the limitations of state-of-the-art vision-and-language models when evaluated on multilingual and multicultural data. First, I will introduce a new protocol to collect culturally relevant images and captions, which resulted in MaRVL, a vision-and-language reasoning dataset in five typologically diverse languages. Then, I will present IGLUE, the first benchmark that evaluates multilingual multimodal models for transfer learning across languages, modalities, and tasks. IGLUE brings together four diverse tasks across 20 diverse languages. Unlike text-only language models, multimodal encoders struggle on zero-shot and few-shot cross-lingual transfer setups.
Bio: Emanuele is a final-year PhD Fellow in the NLP Section at the University of Copenhagen. His research focuses on the intersection of language and vision, with a particular interest in multilinguality and in understanding how models work.
When: 18th Jan
When: 25th Jan 11am