Natural Language Processing (NLP) is the art of teaching a computer how to understand (human) language. It is a type of AI technology that allows an automated system to understand human needs through natural language. Anything that deals with human languages (texts or speeches), such as web search, recommendation systems, chatbots, language translation, and social media, comes under the realm of NLP.
Due to the success of deep learning, in the last decade, NLP has witnessed a massive change in how a computer perceives a language. The blend of classical NLP with neural networks has shown promising performance in almost every application. This crash course intends to provide a holistic view of modern NLP techniques. It will start with two introductory sections -- basics of NLP and introduction to deep learning, following which it will broadly focus on deep learning methods for NLP. The course is designed for senior undergraduate and graduate students of any discipline, who carry introductory knowledge of machine learning and a strong Python programming skill. The students are not expected to carry any prior knowledge of NLP at the start of the course. We will cover foundational architectures, different training protocols, and several downstream NLP tasks. The course will include programming assignments (in Python), quizzes, course projects, research paper reading, and two written exams (midterm and major).
Introduction to classical NLP, introduction to deep learning, unsupervised word vectors, recurrent models, attention and self-attention, Transformers and their variants, transfer learning, prompt-based learning, in-context learning, multilingual NLP, RAG, ethics in NLP.
Previous edition: https://sites.google.com/view/ell881-iitd/
Monday and Thursday, 2:00 PM - 3:30 PM
Office hours: Monday 5:00 PM - 5:30 PM
Classroom: LH 308
Audit Policy: A minimum of B- grade.
Attendance Policy: 70% (Timble).
Grading Scheme: TBD.
Mini-project Release: 11/01/2024
Assignment 1: 25/01/2024
Quiz 1: 15/02/2024
Mid-Term: 19/02/2024 - 24/02/2024
Assignment 2: 20/03/2024
Assignment 3: 01/04/2024
Quiz 2: 25/04/2024
Quiz 3: 15/04/2024 (extra quiz)
Major: 27/04/2024 - 04/05/2024
Mini-project assessment: TBD
For neural networks and deep learning:
Michael A. Nielsen. Neural Networks and Deep Learning
Eugene Charniak. Introduction to Deep Learning
Ian Goodfellow. Yoshua Bengio, and Aaron Courville. Deep Learning
Yoav Goldberg. Neural Network Methods for Natural Language Processing
For classical NLP:
Dan Jurafsky and James H. Martin. Speech and Language Processing
Advanced NLP, Graham Neubig http://www.phontron.com/class/anlp2022/
Advanced NLP, Mohit Ayyer https://people.cs.umass.edu/~miyyer/cs685/
NLP with Deep Learning, Chris Manning, http://web.stanford.edu/class/cs224n/
Understanding Large Language Models, Danqi Chen https://www.cs.princeton.edu/courses/archive/fall22/cos597G/
Self-supervised Learning, Daniel Khashabi, https://self-supervised.cs.jhu.edu/fa2022