Independent Study Class

Objective

The goal of this course is to review the state-of-the-art in representation learning.

Room and Time

453-114 Wednesday 7:30am-9:00am

Schedule

 Session         Paper      Additional material
Sept 18 Representation Learning: A Review and New Perspectives  [video][ppt]
Sept 25 What Regularized Auto-Encoders Learn from the Data Generating Distribution  [video]
Oct 8 Generalized Denoising Auto-Encoders as Generative Models  [video][ppt]
Oct 15-22 Deep Generative Stochastic Networks Trainable by Backprop   [code][video][ppt]
Oct 30Deep Learning of Representations: Looking Forward.  

Resources

Master class -  Yoshua Bengio (21 Oct - 24 Oct 2013)

Bibliography

  • Bengio, Y.; Courville, A.; Vincent, P., "Representation Learning: A Review and New Perspectives," Pattern Analysis and Machine Intelligence, IEEE Transactions on , vol.35, no.8, pp.1798,1828, Aug. 2013, doi: 10.1109/TPAMI.2013.50
  • Guillaume Alain, Yoshua Bengio, "What Regularized Auto-Encoders Learn from the Data Generating Distribution," eprint arXiv:1211.4246, http://arxiv.org/abs/1211.4246v4
  • Bengio, Y., Yao, L., Alain, G., & Vincent, P. (2013). Generalized Denoising Auto-Encoders as Generative Models. CoRR, abs/1305.6663, http://arxiv.org/abs/1305.6663
  • Bengio, Y., & Thibodeau-Laufer, E. (2013). Deep Generative Stochastic Networks Trainable by Backprop. CoRR, abs/1306.1091. http://arxiv.org/abs/1306.1091
  • Bengio, Y. (2013). Deep Learning of Representations: Looking Forward. CoRR, abs/1305.0., http://arxiv.org/abs/1305.0445
  • Practical recommendations for gradient-based training of deep architectures, Yoshua Bengio, U. Montreal, arXiv report:1206.5533, Lecture Notes in Computer Science Volume 7700, Neural Networks: Tricks of the Trade Second Edition, Editors: Grégoire Montavon, Geneviève B. Orr, Klaus-Robert Müller, 2012. http://arxiv.org/abs/1206.5533
Links of interest