Course Objectives

Natural Language Processing (NLP) is the art of teaching a computer how to understand (human) language. It is a type of AI technology that allows an automated system to understand human needs through natural language. Anything that deals with human languages (texts or speeches), such as web search, recommendation systems, chatbots, language translation, and social media, comes under the realm of NLP. 

Due to the success of deep learning, in the last decade, NLP has witnessed a massive change in how a computer perceives a language. The blend of classical NLP with neural networks has shown promising performance in almost every application. This crash course intends to provide a holistic view of modern NLP techniques. It will start with two introductory sections -- basics of NLP and introduction to deep learning, following which it will broadly focus on deep learning methods for NLP. The course is designed for senior undergraduate and graduate students of any discipline, who carry introductory knowledge of machine learning and a strong Python programming skill. The students are not expected to carry any prior knowledge of NLP at the start of the course. We will cover foundational architectures, different training protocols, and several downstream NLP tasks. The course will include programming assignments (in Python), quizzes, course projects, research paper reading, and three written exams (minor-1, minor-2 and major).

Course contents

Introduction to classical NLP, introduction to deep learning, unsupervised word vectors, recurrent models, attention and self-attention, Tranformers and thair variants, transfer learning, prompt-based learning, in-context learning, multilingual NLP, ethics in NLP.

Teaching Assistants

1. Kshitij Alwadhi 

2. Gurusha Juneja 

Class Schedule and Location

• Monday and Thursday, 2:00 - 3:30 PM

• Office hours: Mon 5-6 pm (require prior appoinment by email)

• Classroom: LH-519, LH512

• Group email: 2202-ELL881@courses.iitd.ac.in

Piazza Link

https://piazza.com/iitd.ac.in/spring2023/ell881 

Evaluations Timeline (Tentative)

• Project Release: 18/1/23

• Assignment 1: 21/1/23 - 27/1/23

• Quiz 1: 2/2/23

• Assignment 2: 14/3/23 - 20/3/23

• Quiz 2: 29/3/23

• Assignment 3: 10/4/23 - 23/4/23

• Quiz 3: 24/4/23 17/4/23

• Makeup Quiz: 27/04/23

• Release of the test set for the project: 11/05/23

• Project Evaluation: 12/05/23

Suggested Textbooks

For neural networks and deep learning:

• Michael A. Nielsen. Neural Networks and Deep Learning

• Eugene Charniak. Introduction to Deep Learning

• Ian Goodfellow. Yoshua Bengio, and Aaron Courville. Deep Learning

• Yoav Goldberg. Neural Network Methods for Natural Language Processing

For classical NLP:

• Dan Jurafsky and James H. Martin. Speech and Language Processing

Related Courses (Non-exhaustive)

• Advanced NLP, Graham Neubig http://www.phontron.com/class/anlp2022/

• Advanced NLP, Mohit Ayyer https://people.cs.umass.edu/~miyyer/cs685/

• NLP with Deep Learning, Chris Manning, http://web.stanford.edu/class/cs224n/

• Understanding Large Language Models, Danqi Chen https://www.cs.princeton.edu/courses/archive/fall22/cos597G/

• Self-supervised Learning, Daniel Khashabi, https://self-supervised.cs.jhu.edu/fa2022