Structured Prediction

Instructor: Shashank Srivastava
ssrivastava@cs.unc.edu

When: Tuesday and Thursday, 2:00 pm-3:15 pm ET
Where: SN 011
Office hours: Thursdays, 1-2pm (email for appointment)

Sign up on course Piazza (Access code is comp790)

Announcement: Sign-up for paper presentations here!

Course overview

This is a research-oriented course on structured prediction, which is the class of prediction problems where the output is constrained by some structure. These include labeling, alignment, parsing, ranking and segmentation problems over structures such as sequences, trees and graphs. The main focus of the course will be on problems from natural language processing, although the methods have applications in many domains (computer vision, computational biology, social media analysis, information retrieval, etc.). We will also explore some methods for machine learning with structured inputs.

The course will involve weekly paper readings and a research project. We will explore both recent research and foundational methods in this area. The tentative list of topics that we will cover is:

• Structures: sequences, trees, graphs.

• Practical concerns in predicting structures

• Linear Sequence models: HMMs, MEMMs, CRFs

• Structured Perceptron and Large-margin methods (M3Ns, Structured SVMs)

• Inference methods: Dynamic Programming, Graph algos, ILPs, MCMC Sampling, Variational Inference, Continuous Inference Networks

• Graphical Models, Latent Variable Modeling, VAEs

• Deep Structured Prediction, Energy-based Models

• Graph-based and Transformer-based Neural Networks

• Optimization: SGD, EM, Dual Decomposition, AD3

• Learning to Search and RL: DAgger & SEARN-like methods

Prerequisites:

This course assumes previous experience with Machine Learning (prior exposure to NLP will be a plus), although there are no formal pre-requisites. If you have not taken such classes before or are uncertain, you should speak to me.

Class Schedule (tentative):