Deep Learning (Spring 2021)

Course Title: Deep Learning CSE Program Elective Course (UG3 & UG4) L-T-P-C: 3 - 1 - 0 – 4

Spring 2021

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Course Outline (Topics): The following list of topics is tentative. Based on available time slots, some topics may be dropped or added or reordered.

Artificial Neural Networks: Introduction, Journey from Machine Learning to Deep Learning, Linear Classifiers, Multi-class Classification, Non-linear Classification, Review of Neural Networks, Multilayer Perceptron

Convolutional Neural Networks: CNN, Training Aspects of Neural Networks, Gradient Descent Optimizers, Initialization, Dropout, Batch Normalization, Data Augmentation, Transfer Learning, etc.

CNN Architectures: Image Classification (LeNet, AlexNet, VGG, GoogleNet, ResNet, SENet, ResNeXt, DenseNet), Object Detection (R-CNN, Fast R-CNN, Faster R-CNN, YOLO, SSD, RetinaNet), Segmentation (Mask R-CNN), etc.

Recurrent Neural Networks: RNN, LSTM, Bi- LSTM, GRU, Machine Translation, etc.

Unsupervised Deep Learning: Siamese Networks, Autoencoder Networks, Applications

Generative Adversarial Networks: GAN, Image to Image Translation, Applications

Deep Learning Applications: Chatbot, Speech Recognition, Image Summarization, Visual-Question Answering, Text-Speech and Text-Image Synthesis, etc.

Recent Trends: Deep Reinforcement Learning, Neural Architecture Search, CNN Pruning, Attention Network, Explainable AI, etc.

Grading Policy:

Assignments and term project should include explanatory/clear comments as well as a short report describing the approach, detailed analysis, and discussion/conclusion. Note that the grading policy may change based on the number of registration in the course.

20% Mid-Exam

30% End-Exam

10% Surprise Quizzes

40% Assignments/Projects

Tentative Assignment/Project Schedule:

Assignment 1 - Released: 22-Jan-2021 Deadline: 11-Feb-2021

Assignment 2 - Released: 20-March-2021 Deadline: 10-April-2021

Project Plan Finalization: 05-Feb-2021 & Weekly Reviews

Lecture Slides

Week 1: Lecture 1 (Jan 04, 2021) Introduction PDF, Video

Week 1: Lecture 2-3 (Jan 05,06, 2021) Introduction to statistical learning PDF, Video1, Video2

Week 2: Lecture 4-5 (Jan 11,12, 2021) Linear Classifiers PDF, Video1, Video2

Week 2: Lecture 6 (Jan 15, 2021) Multi-Class Classification PDF, Video, Video-Tute

Week 3: Lecture 7 (Jan 18, 2021) Non-Linear Classification PDF, Video

Week 3: Lecture 8 (Jan 19, 2021) Backpropagation PDF, Video

Week 3: Lecture 9 (Jan 22, 2021) Convolutional Neural Networks PDF, Video

Week 4: Lecture 10 (Jan 25, 2021) Convolutional Neural Networks Video

Week 4: Lecture 11 (Jan 29, 2021) Training Aspects of Neural Networks 1 PDF, Video1

Week 5: Lecture 12,13,14 (Feb 01,02,05, 2021) Training Aspects of Neural Networks 2-4 PDF, Video2, Video3, Video4

Week 6: Lecture 15,16, 17 (Feb 08,09,12, 2021) CNN Architectures PDF, Video1, Video2, Video3

Week 7: Lecture 18,19 (March 01,02, 2021) Object Detection using Deep Learning PDF, Video1, Video2

Week 7: Lecture 20 (March 05, 2021) & Week 8: Lecture 21 (March 08, 2021) Segmentation using Deep Learning PDF, Video1, Video2

Week 8: Lecture 22 (March 15, 2021) Self-supervised Learning PDF, Video

Week 8: Lecture 23 (March 16, 2021) Network Visualization PDF, Video

Week 8: Lecture 24 (March 19, 2021) Fooling Deep Learning Models PDF, Video

Week 9: Lecture 25 (March 19, 2021) Autoencoders PDF, Video

Week 10: Lecture 26 (March 22, 2021), Lecture 27 (March 23, 2021) Generative Adversarial Networks PDF, Video1, Video2

Week 10: Lecture 28 (March 26, 2021) Conditional GAN PDF, Video

Week 10: Lecture 29 (March 26, 2021) Variational Autoencoder (VAE) PDF, Video

Week 11: Lecture 30 (March 30, 2021) Recurrent Neural Network (RNN) PDF, Video

Week 12: Lecture 31 (April 05, 2021) LSTM + GRU PDF, Video

Week 12: Lecture 32 (April 06, 2021) Attention PDF, Video

Week 13: Lecture 33 (April 12, 2021) Self-Attention and Transformer PDF, Video

Week 13: Lecture 34 (April 16, 2021) Deep Reinforcement Learning PDF, Video

Week 13: Lecture 35 (April 16, 2021) Trends in Deep Learning PDF, Video