222EIA001: Deep Learning 


Online Meeting Link

https://meet.google.com/pmp-dybr-mzb 

Syllabus

Introduction - What is Deep Learning? – Machine Learning Vs. Deep Learning, representation Learning, Width Vs. Depth of Neural Networks, Activation Functions: RELU, LRELU, ERELU], Boltzmann, Machines, Auto Encoders. Optimization Techniques, Gradient Descent, Batch, Optimization, Back Propagation - Calculus of Back Propagation,

Bayesian Learning, Decision Surfaces Linear, Classifiers, Machines with Hinge Loss Unsupervised Training of Neural Networks, Restricted Boltzmann Machines, Auto Encoders, Perceptron and Multi-layer Perceptron – Hebbian Learning - Neural net as an Approximator, Training a neural network - Perceptron learning rule - Empirical Risk Minimization - Optimization by gradient descent

Convergence in Neural networks - Rates of Convergence – Loss Surfaces – Learning rate and Data normalization RMSProp, Adagrad and Momentum , Stochastic Gradient Descent Acceleration – Overfitting and Regularization, Choosing a Divergence Loss Function – Dropout – Batch Normalization

Convolutional Neural Networks (CNN) - Weights as Templates – Translation Invariance  Training with shared parameters – Arriving at the convolutional model, Mathematical details of CNN, Alexnet – Inception – VGG - Transfer Learning

Recurrent Neural Networks (RNNs),Modeling sequences - Back propagation through time-Bidirectional RNNs, Exploding/vanishing gradients - Long Short-Term Memory Units (LSTM)

References

CO-PO Mapping

CO-PO Mapping

Evaluation Pattern

Course Plan, Course Coverage and Attendance

Course Plan, Course Coverage and Attendance

Review Paper

Rubrics for Assessment of Review Paper Folder for Uploading Review Paper 

Internals

TBA

Course Materials

Assignment

Assignment 1

General Instructions

Question 1: Implement a 2-input perceptron in any of the scientific computing languages of your choice. Using perceptron learning algorithm, learn the the weights and bias of the separating line so that the perceptron behaves as: 

Question 2: One of the limitations of a perceptron is that it can only learn to discriminate linearly separable data but this limitation  can be overcome by combining two or more perceptrons. Explain how you can implement a system using multiple perceptrons to behave as a:


Assignment 2

General Instructions

Question 1: Given a basic RNN with a single hidden layer consisting of 50 recurrent neurons and an input sequence that lasts for 8 time steps with each input vector being 20-dimensional, how many parameters are necessary for the training of this neural network? [CO2]

Question 2: How is the training process of a Recurrent Neural Network (RNN) facilitated by propagating errors backwards through time? {CO2]

Question 3: How would you characterize a neural network design that interprets sequences from both starting and ending points? {CO2]

Bonus Question: Imagine being in a grand library, surrounded by ancient tomes and manuscripts. You come across a book titled "Legends of the Descent" that narrates the tales of four legendary navigators: Old Sage Gradient, Momentum Max, Adagrad Ada, and RMSProp Rosie. Each has a unique method for descending the mysterious "Valley of Losses." Your curiosity gets the better of you, and you turn to the librarian and ask, "I'm familiar with Old Sage Gradient, who uses the immediate terrain's feel beneath his feet to decide his next move. But what about the others? How do the methods of Momentum Max with his momentum-packed boots, Adagrad Ada with her history-etched map, and RMSProp Rosie with her fusion crystal ball stand out and possibly enhance the age-old descent technique the Old Sage uses?" [CO1]