Summer Semester 2025 - Course

Advanced Natural Language Processing

This course teaches students to understand and apply deep learning methods for natural language processing, with a particular emphasis on large language models (LLMs). Students will spend most of the term exploring neural language models, a key driver in advancing the state of the art in the field. It is designed for Masters students in computer science or informatics who are (1) interested in keeping pace with cutting-edge research developments in NLP and (2) have a solid background in machine learning fundamentals. The course covers key algorithmic foundations and applications of advanced natural language processing. 

Organization

• This seminar is organized by Dr. Jennifer D'Souza.

• The seminar is available for master students of the "Electrical Engineering and Information Technology" or "Computer Science" programs at the Leibniz University of Hannover and at present is offered in Winter Semester 2024/2025 (WiSE 24/25).

• Course location: Raum A141, Gebaeude 3403 (Map: https://info.cafm.uni-hannover.de/de/room/3403.001.A141)

• Course day/time: Every Tuesday, 9am to 10:30am

Class Format

For each class there will be:

• Reading: Most classes will have associated reading material that is recommended you read before the class to familiarize yourself with the topic.

• Lecture and Discussion: There will be a lecture and discussion regarding the class material.

• Code/Data Walk: Some classes will involve looking through code or data. Check https://github.com/jd-coderepos/advanced-nlp-course/ 

• Optional Graded Quiz and Programming Assignments: The homework includes both programming and writing assignments. The course grade will depends on the written exam. The optionally graded homework (15%) and participation (10%) will be added to the final exam as bonus points. 

Acknowledgements

This semester's course is adapted from Advanced NLP Fall 2024, designed by Graham Neubig and taught at  Carnegie Mellon University’s Language Technology Institute . 

Readings

A nice textbook for NLP fundamentals is Jurafsky and Martin, Speech and Language Processing, 3rd ed. For this course, readings will mainly be NLP conference papers (e.g., from ACL, NAACL, and EMNLP). We will post all readings as PDFs. 

Other useful texts for NLP include:

• Eisenstein, Natural Language Processing. Draft textbook.

• Smith, Linguistic Structure Prediction. Free access at UMass. Short book. Excellent coverage of structured prediction inference methods for NLP.

• Murphy, Machine Learning: a Probabilistic Perspective. Excellent, though advanced, coverage of most of the machine learning methods we will use.

• Bender, Linguistic Fundamentals for NLP. Short book. Focuses on linguisic issues relevant to NLP.

• Bird et al, NLP with Python, a.k.a. the NLTK book. Aimed at a more introductory level than this course, but the book is a good gentle introduction to NLP with a CL (computational linguistics) emphasis. The NLTK software has easy-to-use data access and some interfaces to (not always SOTA) NLP tools.