Materials for lecture 5

Post date: Nov 03, 2015 11:9:54 PM

Unfortunately I don't have handouts for backpropagation. However, there are many excellent materials on the Internet:

• 1. Very intuitive and simple:

     1. Cristopher Olah's post http://colah.github.io/posts/2015-08-Backprop/

     2. Michael Nielsen's book http://neuralnetworksanddeeplearning.com/chap2.html

  2. More advanced materials:

• • 1. Chapters from the Deep Learning book by Y. Bengio:

    2. http://www.iro.umontreal.ca/~bengioy/dlbook/numerical.html

    3. http://www.iro.umontreal.ca/~bengioy/DLbook/mlp.html

• 3. Notes from A. Karpathy http://karpathy.github.io/neuralnets/

  4. The gory details:

• • 1. Y. LeCun "Efficient BackProp" http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf

    2. Y. Bengio "Practical recommendations for gradient-based training of deep architectures" http://arxiv.org/pdf/1206.5533v2.pdf

For network visualization and intuitions please see:

• 1. Cristopher Olah's post http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/

  2. ConvNetJS demo: http://cs.stanford.edu/people/karpathy/convnetjs/