Reference Material

There are many manuscripts for deep learning theory now. Many things in first half of the course will be based on this manuscript: https://www.cs.princeton.edu/~arora/TheoryDL.pdf

The materials on initialization can be found on Matus Telgarsky's manuscript https://mjt.cs.illinois.edu/dlt/two.pdf

Other deep learning theory related books include https://issuu.com/cmb321/docs/deep_learning_ebook and https://arxiv.org/abs/2106.10165 (and I'm sure there are books that I haven't read.)

Additional materials will be posted as lectures go on.