Readings

Probability theory  (by Matthew Shum)

Introduction to probability  by C.M. Grinstead and J.L. Snell

Linear algebra review  

Learning  Python

Python tutorial for deep learning

Math and Matrix Operations to Python

Deep Learning:  Linear Algebra

Deep Learning:  Machine Learning Basics

Vectors (by 3Blue1Brown)

Overview of differential equations   (by 3Blue1Brown)

Linear transformations and matrices ( by 3Blue1Brown)

COGS118A

• Part I (One-hot encoding, Vector calculus, Decision boundary, Decision stump, Estimation, Convexity, Linear regression)

• Part II (Error metrics, Perceptron, Logistic regression, Support vector machine, Complexity, Corss-validation)

COGS 118A

Part I (One-hot encoding, Vector calculus, Decision boundary, Decision stump, Estimation, Convexity, Linear regression)

Part II (Error metrics, Perceptron, Logistic regression, Support vector machine, Complexity, Cross-validation)

Convex function

Matrix calculus

Deep Learning:  Numeric Optimization

Perceptron

Logistic regression

Logistic Regression (Andrew Ng)

Gradient descent 

UC Irvine ML: Gradient Descent (Alexander Ihler) 

Stanford CS231n: Lecture 4:  Introduction to Neural Networks 

Deep feedforward neural networks  (I. Goodfellow, Y. Bengio, and A. Courville)

Feedforwad neural networks

Neural Networks  (3Blue1Brown) 

Backpropagation

Circuit illustration of backpropagation

Gradient descent, how neural networks learn  (3Blue1Brown)

Convolution

Convolution: a demo

Stanford CS231n:  Lecture 5, Convolutional Neural Networks 

Illustrations for Feedforward Neural Networks: YouTube (by 3Blue1Brow)

Convolutional networks  (I. Goodfellow, Y. Bengio, and A. Courville) 

Stanford CS231n:  Lecture 6, Training Neural Networks I 

Regularization for Deep Learning  (I. Goodfellow, Y. Bengio, and A. Courville)

Optimization for Training DeepModels  (I. Goodfellow, Y. Bengio, and A. Courville) 

Stanford CS231n: Lecture 7,  Training Neural Networks II 

Illustration for Convolution: YouTube (by 3Blue1Brown)

AlexNet, GoogLeNet, VGG, ResNet, ResNeXt

Stanford CS231n: Lecture 9, CNN Architectures 

 Recurrent Neural Networks  (I. Goodfellow, Y. Bengio, and A. Courville) 

Stanford CS231n: Lecture 10,  Recurrent Neural Networks 

Hopfield Neural Network

J. J. Hopfield and D. W. Tank,  “Neural” computation of decisions in optimization problems , Biological Cybernetics 1985. 

Attention Is All You Need. link

BERT, link

ViT, link

GAN from scratch

Deep Generative Models (I. Goodfellow, Y. Bengio, and A. Courville)

Autoencoders  (I. Goodfellow, Y. Bengio, and A. Courville)

Variational autoencoder 

Stanford CS231n: Lecture 13, Generative Models 

Stanford CS231n: Lecture 16, Adversarial Examples and Adversarial Training