IFT6135-H20 DIRO, Université de Montréal

Course Description:

This is a course on representation learning in general and deep learning in particular. Deep learning has recently been responsible for a large number of impressive empirical gains across a wide array of applications including most dramatically in object recognition and detection in images and speech recognition.

In this course we will explore both the fundamentals and recent advances in the area of deep learning. Our focus will be on neural network-type models including convolutional neural networks and recurrent neural networks such as LSTMs. We will also review recent work on attention mechanism and efforts to incorporate memory structures into neural network models. We will also consider some of the modern neural network-base generative models such as Generative Adversarial Networks and Variational Autoencoders.

The course will use the textbook: Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville (available for order at amazon or online for free here).

In particular topics covered by the course include the following:

• Neural networks structure and training algorithms
• Convolutional neural networks
• Recurrent neural networks
• Transformer Networks
• Optimization methods for training neural networks
• Regularization of neural networks
• Normalization Methods (Batch Normalization, etc.)
• Attention Mechanism
• Applications to Computer Vision tasks (Image Segmentation, Bounding Box Detection, etc.)
• Applications to Natural Language Processing tasks
• Meta-Learning
• Autoencoders
• Variational Autoencoders
• Generative Adversarial Networks (GANs)
• Autoregressive Generative Models
• Neural Flow-based Generative Models
• RBMs and DBMs (if time permits)

Instruction style: The lecture time will be devoted to traditional lectures. Students are responsible for keeping up-to-date with the course material outside of class time, mainly by reading the textbook and other readings assigned for each lecture. The material to be reviewed for each class will be made available on this course website.

Evaluation:

The course will include a set of 3 assignments. Each assignment will consist of three components:

1. A problem set of theoretical questions
2. A practical component involving programming and running experiments.
3. An in-class quiz on the material covered in both the theoretical and practical components of the assignment (with an emphasis on the theoretical component).

See the Assignment Policy below for more information. The in-class quiz will take place on the due date / or at the first class after the due date of each assignment. Each quiz will take 20 minutes to finish.

In addition to these assignments there will be a final exam. Both the practical and theoretical component of the assignments is to be done individually.

The final grade will be composed as follows:

• Assignments: 3 * 25% = 75% (For each assignment, 25% = 10% (theory) + 10% (practical) + 5% (quiz)
• Final Exam: 25% - Final exam is 3 hours long and covers all material seen in class and in the assignments. Students are permitted a one page (two-sided) hand written cheat sheet 8.5"x11".

Instructors:

• Instructor: Prof. Aaron Courville
• Teaching assistants:
  • Chin-Wei Huang (chin-wei.huang at umontreal.ca),
  • Sai Rajeshwar (rajsai24 at gmail.com),
  • Samuel Lavoie (Samuel.Lavoie.m at gmail.com) -- questions en français,
  • Sanae Lotfi (sanae.lotfi@polymtl.ca) -- questions en français,
  • Akilesh Badrinaaraayanan (akilesh041195 at gmail.com),
  • Lluís Castrejón Subirà (lluis.castrejon.subira at gmail.com),
  • Jae Hyun Lim (jae.hyun.lim at umontreal.ca),
  • Jessica Thompson (j.thompson at umontreal.ca),
  • Jie Fu (jie.fu at polymtl.ca),
  • Jonathan Cornford (cornforj at mila.quebec),
  • Michael Noukhovitch (mnoukhov at gmail.com) -- GradeScope Specialist

Previous Exams:

Warning: material changes from year to year. This year's exam will include a short answer / true-false section, approx. 50% of the exam.)

• IFT6135-H2018 Final Exam
• IFT6266 2017 Final Exam (français)
• IFT6266 2015 Final Exam
• IFT6266 2015 Quiz