Abstract: Most current work in natural language understanding is based on training sets which only contain text and possibly associated labels. We hypothesize that this is insufficient to generally capture meaning, even with very large corpora, because a lot of meaning is associated with hard to verbalize knowledge about the world, associated in cognitive psychology with system 1 abilities, perception and motor abilities and intuitive understanding. On the other hand, much of the work on unsupervised representation learning from low-level data like images or video assumes that the high-level factors are marginally independent. We propose an approach in which high-level semantic variables which have a direct connection with language have a causal interpretation with strong causal dependencies between them and are grounded in low-level perception and action via system 1 computation. System 2 abilities like reasoning, planning and credit assignment (explaining observations) rely on attention mechanisms and modular knowledge decomposition favouring out-of-distribution and systematic generalization. The talk will report on preliminary work in these directions, early steps in a long-term research program.

Bio: Yoshua Bengio is recognized as one of the world’s leading experts in artificial intelligence and a pioneer in deep learning. Since 1993, he has been a professor in the Department of Computer Science and Operational Research at the Université de Montréal. He is the founder and scientific director of Mila, the Quebec Institute of Artificial Intelligence, the world’s largest university-based research group in deep learning. He is a member of the NeurIPS board and co-founder and general chair for the ICLR conference, as well as program director of the CIFAR program on Learning in Machines and Brains and is Fellow of the same institution. In 2018, Yoshua Bengio ranked as the computer scientist with the most new citations, worldwide, thanks to his many publications. In 2019, he received the Killam prize as well as the ACM A.M. Turing Award, “the Nobel Prize of Computing”, jointly with Geoffrey Hinton and Yann LeCun for conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing. In 2020 he was nominated Fellow of the Royal Society of London.

Link: https://www.youtube.com/watch?v=u3IR6sSwwjg

Slides: https://drive.google.com/file/d/1-B6Afb2XCBHszW98T5KcqVFMF4akFs4x/view?usp=sharing

Abstract: Neural language models, as they grow in scale, continue to surprise us with utterly nonsensical and counterintuitive errors despite their otherwise remarkable performances on leaderboards. In this talk, I will argue that it is time to challenge the currently dominant paradigm of task-specific supervision built on top of large-scale self-supervised neural networks. I will first highlight how we can make better lemonade out of neural language models by shifting our focus on unsupervised, inference-time algorithms. I will demonstrate how unsupervised algorithms can match or even outperform supervised approaches on hard reasoning tasks such as nonmonotonic reasoning (such as counterfactual and abductive reasoning), or complex language generation tasks that require logical constraints. Next, I will highlight the importance of melding explicit and declarative knowledge encoded in symbolic knowledge graphs with implicit and observed knowledge encoded in neural language models. I will present COMET, Commonsense Transformers that learn neural representation of commonsense reasoning from a symbolic commonsense knowledge graph, and Social Chemistry 101, a new conceptual formalism, a knowledge graph, and neural models to reason about social, moral, and ethical norms.

Bio: Yejin Choi is a Brett Helsel associate professor at the Paul G. Allen School of Computer Science & Engineering at the University of Washington and also a senior research manager at AI2 overseeing the project Mosaic. Her research interests include commonsense knowledge and reasoning, neural language (de-)generation, language grounding, and AI for social good. She is a co-recipient of the AAAI Outstanding Paper Award in 2020, Borg Early Career Award (BECA) in 2018, IEEE’s AI Top 10 to Watch in 2015, the ICCV Marr Prize in 2013, and the inaugural Alexa Prize Challenge in 2017.

Link: https://www.youtube.com/watch?v=QN_LUgU9-kg

Slides: drive.google.com/file/d/1hRI3so28CYtxcx96UughdZEviZQ_RrmE/view?usp=sharing

Abstract: In many types of human learning, task descriptions are a central ingredient. They are usually accompanied by a few examples, but there is very little human learning that is based on examples only. In contrast, the typical learning setup for NLP tasks lacks task descriptions and is supervised with 100s or 1000s of examples. This is even true for so-called few-shot learning, a term often applied to scenarios with tens of thousands of "shots". Inspired by the GPT models, which also exploit task descriptions, we introduce Pattern-Exploiting Training (PET). PET reformulates task descriptions as cloze questions that can be effectively processed by pretrained language models. In contrast to GPT, PET combines task descriptions with supervised learning. We show that PET learns well from as little as ten training examples and outperforms GPT-3 on GLUE even though it has 99.9% fewer parameters.

Bio: Hinrich Schütze (PhD 1995, Stanford University) is Professor for Computational Linguistics and director of the Center for Information and Language Processing at the University of Munich (LMU Munich). Before moving to Munich in 2013, he taught at the University of Stuttgart. He worked on natural language processing and information retrieval technology at Xerox PARC, at several Silicon Valley startups and at Google 1995-2004 and 2008/9. He is a coauthor of Foundations of Statistical Natural Language Processing (with Chris Manning) and Introduction to Information Retrieval (with Chris Manning and Prabhakar Raghavan). He was awarded a European Research Council Advanced Grant in 2017. Hinrich serves as action editor for TACL and is currently the president of the Association for Computational Linguistics.

Link: https://www.youtube.com/watch?v=_YOaRLQnBjc

Abstract: Current evaluation of the generalization of natural language processing (NLP) systems, and much of machine learning, primarily consists of measuring the accuracy on held-out instances of the dataset. Since the held-out instances are often gathered using similar annotation process as the training data, they include the same biases that act as shortcuts for machine learning models, allowing them to achieve accurate results without requiring actual natural language understanding. Thus held-out accuracy is often a poor proxy for measuring generalization, and further, aggregate metrics have little to say about where the problem may lie. In this talk, I will introduce a number of approaches we are investigating to perform a more thorough evaluation of NLP systems. I will first provide a quick overview of automated techniques for perturbing instances in the dataset that identify loopholes and shortcuts in NLP models, including semantic adversaries and universal triggers. I will then describe recent work on creating comprehensive and thorough tests and evaluation benchmarks for NLP using CheckList, that aim to directly evaluate comprehension and understanding capabilities. The talk will cover a number of NLP tasks, including sentiment analysis, textual entailment, paraphrase detection, and question answering.

Bio: Sameer Singh is an Assistant Professor of Computer Science at the University of California, Irvine (UCI). He is working primarily on the robustness and interpretability of machine learning algorithms, along with models that reason with text and structure for natural language processing. Sameer was a postdoctoral researcher at the University of Washington and received his Ph.D. from the University of Massachusetts, Amherst, during which he also worked at Microsoft Research, Google Research, and Yahoo! Labs. He was selected as a DARPA Riser, and has been awarded the grand prize in the Yelp dataset challenge, the Yahoo! Key Scientific Challenges, UCI Mid-Career Excellence in research award, and recently received the Hellman Fellowship. His group has received funding from Allen Institute for AI, Amazon, NSF, DARPA, Adobe Research, Base 11, and FICO. Sameer has published extensively at machine learning and natural language processing conferences and workshops, including paper awards at KDD 2016, ACL 2018, EMNLP 2019, AKBC 2020, and ACL 2020.

Link: https://www.youtube.com/watch?v=ViXkQjEqKKo

Slides: drive.google.com/file/d/1RtDNpuhSa6p_1PhVKUOTxU4dy-M1Slt-/view?usp=sharing

Abstract: Much NLP literature has examined social biases in datasets, algorithms, or model performance, and the negative pipeline between them: models absorb and amplify data biases, which causes representational harms and impacts performance. In this talk, I will present studies that look further up the pipeline and rely on the assumption that biases in data originate in human cognition. I will discuss several lightly supervised, interpretable approaches—grounded in social psychology and causal reasoning—to detect implicit social bias in written discourse and narrative text. Together, these approaches aim at providing people-centered text analytics, to proactively pinpoint and explain potentially biased framings—across languages, data domains, and social contexts—before these biased framings propagate into downstream models.

Bio: Yulia Tsvetkov is an assistant professor at Carnegie Mellon University, and in 2021 she will be joining the Paul G. Allen School of Computer Science & Engineering at University of Washington. Her research projects focus on NLP for social good, multilingual NLP and language generation, and are motivated by a unified goal: to extend the capabilities of human language technology beyond individual cultures and across language boundaries, thereby enabling NLP for diverse and disadvantaged users, the users that need it most. Yulia is a recipient of the Okawa research award, Amazon machine learning research award, Google faculty research award, and several NSF awards.

Link: https://www.youtube.com/watch?v=kr8pIwyEe6E

Slides: drive.google.com/file/d/1bdtdOybGrNLNDNGxugVUgF551Dsfl78C/view?usp=sharing