Abstract: Despite major recent advances in cross-linguistic transfer and translation, there is still little understanding of how and when the properties of the source and target language and their divergence and convergence patterns, allow for effective learning. For example, do source-side constructions that do not have an exact parallel on the target side translate less well? The use of semantic and syntactic parsers can address this gap and enable a more fine-grained assessment of the successes and failures of current methods; they can also abstract away from some differences in the way languages express similar content. I will present ongoing work on the use of such parsers to obtain a construction-level picture of system performance, to identify cases where cross-linguistic divergences lead to reduced performance, and to mitigate such challenging divergences by offering a common level of representation between languages.

Bio: Omri Abend (PhD 2014, Hebrew University of Jerusalem) is an associate professor at the Hebrew University, working in the departments of Computer Science and Cognitive Science. His research focuses on semantic representation from a cross-linguistic perspective and semantic parsing, as well as the integration of symbolic semantic methods with statistical methods for NLP and for the evaluation of NLP models. Omri is a co-developer of the UCCA scheme for semantic representation, one of the participating schemes in the CoNLL 2019 and CoNLL 2020 shared tasks on semantic parsing. Omri will serve as one of the two co-chairs for CoNLL 2021, and as an area chair for other major NLP conferences. He regularly serves as a reviewer in the major venues in the field (including TACL). Other than his academic activity, Omri is also employed as a research scientist in AI21 Labs.

Slides: drive.google.com/file/d/128M445F8PmMb9cGYLXLVL0P0nds98crb/view?usp=sharing

Abstract: A recent focus in machine learning and natural language processing is on models that generalize beyond their training distribution. One natural form of such generalization, which humans excel in, is compositional generalization: the ability to generalize at test time to new unobserved compositions of atomic components that were observed at training time. Recent work has shown that current models struggle to generalize in such scenarios. In this talk, I will present recent work, which demonstrates how an inductive bias towards tree structures substantially improves compositional generalization in two question answering setups. First, we present a model that given a compositional question and an image, constructs a tree over the input question and answers the question from the root representation. Trees are not given at training time and are fully induced from the answer supervision only. We show that our approach improves compositional generalization on the CLOSURE dataset from 72.2-->96.1 accuracy, while obtaining comparable performance to models such as FILM and MAC on human-authored questions. Second, we present a span-based semantic parser, which induces a tree over the input to compute an output logical form, handling a certain sub-class of non-projective trees. We evaluate this on several compositional splits of existing datasets, improving performance, on Geo880 e.g., from 54.0-->82.2. Overall, we view these results as strong evidence that an inductive bias towards tree structures dramatically improves compositional generalization compared to existing approaches.

Bio: Jonathan Berant is an associate professor at the School of Computer Science at Tel Aviv University and a research scientist at The Allen Institute for AI. Jonathan earned a Ph.D. in Computer Science at Tel-Aviv University, under the supervision of Prof. Ido Dagan. Jonathan was a post-doctoral fellow at Stanford University, working with Prof. Christopher Manning and Prof. Percy Liang, and subsequently a post-doctoral fellow at Google Research, Mountain View. Jonathan Received several awards and fellowships including The Rothschild fellowship, The ACL 2011 best student paper award, EMNLP 2014 best paper award, and NAACL 2019 best resource paper award, as well as several honorable mentions. Jonathan is currently an ERC grantee.

Slides: drive.google.com/file/d/1RDyAKRcJ83YdWgZmjhRZePWWdRziNvxQ/view?usp=sharing

Abstract: As a simple search heuristic, beam search has been used to decode models developed by the NLP community for decades. Indeed, it is noteworthy that beam search is one of the few NLP algorithms that has stood the test of time: It has remained a cornerstone of NLP systems since the 1970s (Reddy, 1977). As such, beam search became the natural choice for decoding neural probabilistic text generators—whose design makes evaluating the full search space impossible While there is no formal guarantee that beam search will return—or even approximate—the highest-scoring candidate under a model, it has repeatedly proven its merit in practice and, thus, has largely been tolerated—even embraced—as NLP’s go-to search heuristic. This talk further embraces beam search. We discuss two novel formal insights into beam search. In the first act, we discuss an algorithmic advance that allows beam search to be prioritized, i.e. it returns the best hypothesis (modulo the beam size) first. Our algorithmic extension yields a Dijkstra-ified beam search that provably emulates standard beam search. In the second act, we draw a connection between the uniform information density hypothesis from cognitive science and beam search’s efficacy as a search heuristic. We offer a linguistic reason why beam search may work so well in practice even though, as an approximation to the argmax, it may be arbitrarily bad. The work described in this talk is described in publications at TACL (2020) and EMNLP (2020) and won an honorable mention for best paper at the latter.

Bio: Ryan Cotterell (PhD 2019, Johns Hopkins University) is an Assistant Professor of computer science at ETH Zürich. He is also affiliated with the Computer Laboratory (Department of Computer Science and Technology) at the University of Cambridge where he was an Assistant Professor (Lecturer in the UK system) from 2018 to 2020. His research papers have received best-paper awards at ACL 2017 and EACL 2017 and his papers were runners-up for best paper at NAACL 2016, EMNLP 2016, and ACL 2019. During his Ph.D., he was awarded fellowships from Facebook, Fulbright, DAAD, and NDSEG. He was also awarded a Jelinek fellowship at Johns Hopkins [https://www.clsp.jhu.edu/about/jelinek-fellowship/]. He is routinely an area chair in NLP-related venues and is one of the publication chairs for ACL 2020. He serves on the executive boards of SIGTYP and SIGMORPHON, both ACL-affiliated special interest groups. At ETH Zürich, his group is jocularly called Rycolab.

Abstract: In this talk I will sketch three research lines, and the single encompassing goal that triggered them: supporting effective human comprehension, and automatic analysis, of multi-text information. The first line addresses decomposing textual information into explicit natural-language based units, suggesting a mid-way between formal semantic decomposition approaches and implicit distributional representations. Specifically, we propose decomposing a text to the set of question-answer pairs that jointly convey its meaning. Next, we present approaches for aligning corresponding information units across texts, either QA pairs or proposition spans, loosely extending the notion of cross-document coreference resolution to the propositional level. Finally, we advocate that human consumption of multi-text information should be supported by interactive methods. To that end, we propose a systematic and replicable evaluation framework for interactive summarization, enabling principled future research on this task, and further develop a concrete modeling approach framed as Query-focused Summary Expansion. We advocate the appeal of each research line on its own, while, taken together, they aim to address the long awaiting challenge of facilitating human consumption of multi-text information.

Bio: Ido Dagan is a Professor at the Department of Computer Science at Bar-Ilan University, Israel, the founder of the Natural Language Processing (NLP) Lab at Bar-Ilan, the founder and head of the nationally-funded Bar-Ilan University Data Science Institute and a Fellow of the Association for Computational Linguistics (ACL). His interests are in applied semantic processing, focusing on textual inference, natural open semantic representations, consolidation and summarization of multi-text information, and interactive text summarization. Dagan and colleagues initiated textual entailment recognition (RTE, aka NLI) as a generic empirical task. He was the President of the ACL in 2010 and served on its Executive Committee during 2008-2011. In that capacity, he led the establishment of the journal Transactions of the Association for Computational Linguistics, which became one of two premiere journals in NLP. Dagan received his B.A. summa cum laude and his Ph.D. (1992) in Computer Science from the Technion. He was a research fellow at the IBM Haifa Scientific Center (1991) and a Member of Technical Staff at AT&T Bell Laboratories (1992-1994). During 1998-2003 he was co-founder and CTO of FocusEngine and VP of Technology of LingoMotors, and has been regularly consulting in the industry. His academic research has involved extensive industrial collaboration, including funds from IBM, Google, Thomson-Reuters, Bloomberg, Intel and Facebook, as well as collaboration with local companies under funded projects of the Israel Innovation Authority.