Deep Learning

Course Title: Deep Learning CSE Program Elective Course (UG3 & UG4) L-T-P-C: 3 - 1 - 0 – 4

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

0. Pre-requisite: Knowledge of basic computer science principles and skills, at a level sufficient to write a reasonably non-trivial computer program. Familiarity with the probability theory. Familiarity with linear algebra. Machine Learning course is good to have done.

1. Outline: Nowadays, the use of deep learning based information technology is growing exponentially. Most of the big IT companies like Google, Microsoft, Amazon, Facebook, etc. are working over the deep learning. Many startups also came in recent years in deep learning area. It is being used in wide range of applications; including Computer Vision, Natural Language Processing, Robotics and Industrial Automation. It can be also utilized very effectively in smart transportation, manufacturing, medical field, biometrics area, etc.

2. Objectives: Deep Learning is one of the most highly sought after skills in AI. In this course, student will learn the foundations of Deep Learning, understand how to build neural networks, and learn how to lead successful machine learning projects. Specifically, student will learn about CNNs, RNNs, LSTM, Adam, Dropout, BatchNorm, Xavier/He initialization, and more.

3. Course Outline (Topics): The following list of topics is tentative. Based on available time slots, some topics may be dropped or added or reordered.

Artificial Neural Networks: Introduction, Journey from Machine Learning to Deep Learning, Linear Classifiers, Multi-class Classification, Non-linear Classification, Review of Neural Networks, Multilayer Perceptron

Convolutional Neural Networks: CNN, Training Aspects of Neural Networks, Gradient Descent Optimizers, Initialization, Dropout, Batch Normalization, Data Augmentation, Transfer Learning, etc.

CNN Architectures: Image Classification (LeNet, AlexNet, VGG, GoogleNet, ResNet, SENet, ResNeXt, DenseNet), Object Detection (R-CNN, Fast R-CNN, Faster R-CNN, YOLO, SSD, RetinaNet), Segmentation (Mask R-CNN), etc.

Recurrent Neural Networks: RNN, LSTM, Bi- LSTM, GRU, Machine Translation, etc.

Unsupervised Deep Learning: Siamese Networks, Autoencoder Networks, Applications

Generative Adversarial Networks: GAN, Image to Image Translation, Applications

Deep Learning Applications: Chatbot, Speech Recognition, Image Summarization, Visual-Question Answering, Text-Speech and Text-Image Synthesis, etc.

Recent Trends: Deep Reinforcement Learning, Neural Architecture Search, CNN Pruning, Attention Network, Explainable AI, etc.

4. Books/References:

Ian Goodfellow and Yoshua Bengio and Aaron Courville, Deep Learning, MIT Press, 2016

Michael Nielsen, Neural Networks and Deep Learning, 2016

Yoshua Bengio, Learning Deep Architectures for AI, 2009

Charu C. Aggarwal, Neural Networks and Deep Learning: A Textbook, Springer, 2019

François Chollet, Deep Learning with Python, Manning Publications, 2017

Rowel Atienza, Advanced Deep Learning with Keras, Packt Publishing, 2018

Christopher Bishop, Neural Networks for Pattern Recognition, Oxford University Press, 1995.

Christopher Bishop, Pattern Recognition and Machine Learning. Springer, 2006

Richard Duda, Peter Hart, and David Stork, Pattern Classification, 2nd ed. Wiley-Interscience, 2000

6. Course Ethics:

Please note down the following activities leading to a fair academic honesty:

• All class work is to be done independently until not specified.

• It is best to try to solve problems on your own, since problem solving is an important component of the course, and exam problems are often based on the outcome of the assignment problems.

• You are allowed to discuss class material, assignment problems, and general solution strategies with your classmates. But, when it comes to formulating or writing solutions you must work alone.

• You may use free and publicly available sources, such as books, journal and conference publications, and web pages, as research material for your answers. (You will not lose marks for using external sources.)

• You may not use any paid service and you must clearly and explicitly cite all outside sources and materials that you made use of.

• I consider the use of uncited external sources as portraying someone else's work as your own, and as such it is a violation of the Institute's policies on academic dishonesty.

• Instances will be dealt with harshly and typically result in a failing course grade.