Schedule

Topics listed in the schedule are tentative and subject to change

• 01/10/2022 (RA) Introduction

  • Jurafsky and Martin, Speech and Language Processing (3rd edition draft), Chapter 2: Regular Expressions, Text Normalization, and Edit Distance

• 01/12/2022 (RA) Naive Bayes, Linear Classifier

  • Manning, Raghavan and Schutze, Introduction to information retrieval, Chapter 13: Text classification and Naive Bayes

  • Jurafsky and Martin, Speech and Language Processing (3rd edition draft), Chapter 4: Naive Bayes Classification and Sentiment Analysis

  • HW1 Release

• 01/24/2022 (MC) Word Embeddings

  • Mikolov, Tomas, et al. "Efficient estimation of word representations in vector space." arXiv preprint arXiv:1301.3781 (2013)

  • Mikolov, Yih and Zweig (2013): Linguistic Regularities in Continuous Space Word Representations

  • Jurafsky and Martin, Speech and Language Processing (3rd edition draft), Chapter 6: Vector Semantics and Embeddings

• 01/26/2022 (MC) Word Embeddings (continued), Neural Networks & Neural Language Models

  • Jurafsky and Martin, Speech and Language Processing (3rd edition draft), Chapter 7: Neural Networks and Neural Language Models

  • HW1 Deadline

• 02/07/2022 (RA) Sequence Labeling & HMMs

  • Jurafsky and Martin, Speech and Language Processing (3rd edition draft), Chapter 8: Part-of-Speech Tagging

  • Jurafsky and Martin, Speech and Language Processing (3rd edition draft), Appendix A: Hidden Markov Models

  • Michael Collins: Tagging Problems, and Hidden Markov Models

  • HW2 Release

• 02/09/2022 (RA) Discriminative and Unsupervised Sequence Labeling

  • Brill (1992): A Simple Rule-Based Part of Speech Tagger

  • Ratnaparkhi (1996): A Maximum Entropy Model for Part-Of-Speech Tagging

  • Lafferty, McCallum and Pereira (2001): Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

  • Michael Collins: Log-Linear Models, MEMMs, and CRFs

  • Grenager, Klein, and Manning (2005): Unsupervised Learning of Field Segmentation Models for Information Extraction

  • Michael Collins: The Naive Bayes Model, Maximum-Likelihood Estimation, and the EM Algorithm

• 02/21/2022 No class: President’s Day

• 02/23/2022 (MC) Subword representation, contextualized representation

  • https://d2l.ai/chapter_natural-language-processing-pretraining/subword-embedding.html

  • HW2 Deadline

• 03/07/2022 (Guest lecture: Xuezhe Ma) Sequence-to-sequence models, NMT

  • Jurafsky and Martin, Speech and Language Processing (3rd edition draft), Chapter 10: Machine Translation and Encoder-Decoder Models

  • Sutskever, Vinyals and Le (2014): Sequence to Sequence Learning with Neural Networks

• 03/09/2022 (MC) Transformers

  • Attention is All You Need, BERT, GPT2

  • Project Proposal Deadline

  • 03/11/2022: Paper presentation slides due

    • Upload the PDF version of your slides to the Google Drive ( https://drive.google.com/drive/folders/17MwpmS7_GXZxOBu0JjcCBzAignqfJAxX?usp=sharing) and insert the link to your slides in the group assignment sheet column "paper presentation" -> "Link to slides".

• 03/21/2022 Paper presentations (on Zoom only)

• 03/23/2022 Paper presentations (on Zoom only)

  • HW3 Deadline

• 04/04/2022 Question Answering (back to hybrid)

  • UnifiedNLI, UnifiedQA, BART

• 04/06/2022 (MC) Information Extraction

• 04/18/2022 (Jieyu Zhao) Fairness in NLP

  • Project Slides Due

• 04/20/2022 Project presentations

  • HW4 Deadline (extended from 04/13)

• 05/06/2022 Final exam due (online)