IFT 6135 - Representation Learning
Winter 2019, A Deep Learning Course offered by the Université de Montréal
IFT6135-H19 DIRO, Université de Montréal
This is a course on representation learning in general and deep learning in particular. Deep learning has recently been responsible for a large number of impressive empirical gains across a wide array of applications including most dramatically in object recognition and detection in images and speech recognition.
In this course we will explore both the fundamentals and recent advances in the area of deep learning. Our focus will be on neural network-type models including convolutional neural networks and recurrent neural networks such as LSTMs. We will also review recent work on attention mechanism and efforts to incorporate memory structures into neural network models. We will also consider some of the modern neural network-base generative models such as Generative Adversarial Networks and Variational Autoencoders.
In particular topics covered by the course include the following:
- Neural networks structure and training algorithms
- Convolutional neural networks
- Recurrent neural networks
- Optimization methods for training neural networks
- Regularization of neural networks
- Normalization Methods (Batch Normalization, etc.)
- Attention Mechanism
- Memory Models
- Transformer Networks
- Variational Autoencoders
- Generative Adversarial Networks (GANs)
- Autoregressive Generative Models
- RBMs and DBMs (if time permits)
Instruction style: The lecture time will be devoted to traditional lectures. Students are responsible for keeping up-to-date with the course material outside of class time, mainly by reading the textbook and other readings assigned for each lecture. The material to be reviewed for each class will be made available on this course website.
Evaluation: The course will include a set of 3 assignments. Each assignment will consist of two components:
- A problem set of theoretical questions
- A practical component involving programming and running experiments.
In addition to these assignments there will be a final exam. The practical components of assignments can be done in groups of 3 or 4 students. The theoretical component of the assignments is to be done individually.
The final grade will be composed as follows:
- Assignments: 3 * 25% = 75% (For each assignment, 25% = 10% (theory) + 15% (practical)
- Final Exam: 25% (Final exam is closed book, 2 hours long and covers all material seen in class and in the assignments.)
- Instructor: Prof. Aaron Courville
- Teaching assistants:
- Chin-Wei Huang (chin-wei.huang at umontreal.ca ),
- Sai Rajeshwar (rajsai24 at gmail.com)
- Shawn Tan (tanjings at iro.umontreal.ca)
- Samuel Lavoie (Samuel.Lavoie.m at gmail.com)
- Tegan Maharaj (firstname.lastname@example.org)
- David Krueger (david.scott.krueger at gmail.com)
- Salem Lahlou (salemlahlou9 at gmail.com)
- Michael Noukhovitch (mnoukhov at gmail.com)
- Mondays: 9:30 – 11:30 AM (B-0215 Pav. 3200 J.-Brillant)
- Wednesdays: 12:30 – 2:30 PM (B-0245 Pav. 3200 J.-Brillant)
- NOTE EXCEPTION: On Wednesday Feb. 27th, 2019, the course will be 12h30 à 14h30, salle E-310, pavillon Roger Gaudry.
- Prof: Mondays: 11:30 – 12:30 AM (B-0215 Pav. 3200 J.-Brillant - tentative)
- TAs: (TBA) location: Pav Andre Aisenstadt 3336
- 1st Assignment Due: 2/16 23:59 (Kaggle) and 2/17 23:59 (reports)
- 2nd Assignment Due: March 22nd 23:59, 2019
- 3rd Assignment Due: Theory: April 5th 23:59, 2019, Practical: April 19th 23:59, 2019
- Final Exam: last day of class - 10/04/2019 (location: meet at Pavillon Claire-McNichol Z-317, we will be in Z-317, Z-200 and Z-209)
Warning: material changes from year to year. This year's exam will include a short answer / true-false section, approx. 50% of the exam.)