https://meet.google.com/pmp-dybr-mzb
Module 1: Introduction to Deep Learning
Introduction - What is Deep Learning? – Machine Learning Vs. Deep Learning, representation Learning, Width Vs. Depth of Neural Networks, Activation Functions: RELU, LRELU, ERELU], Boltzmann, Machines, Auto Encoders. Optimization Techniques, Gradient Descent, Batch, Optimization, Back Propagation - Calculus of Back Propagation,
Module 2: Neural Networks
Bayesian Learning, Decision Surfaces Linear, Classifiers, Machines with Hinge Loss Unsupervised Training of Neural Networks, Restricted Boltzmann Machines, Auto Encoders, Perceptron and Multi-layer Perceptron – Hebbian Learning - Neural net as an Approximator, Training a neural network - Perceptron learning rule - Empirical Risk Minimization - Optimization by gradient descent
Module 3: Convergence in Neural networks
Convergence in Neural networks - Rates of Convergence – Loss Surfaces – Learning rate and Data normalization RMSProp, Adagrad and Momentum , Stochastic Gradient Descent Acceleration – Overfitting and Regularization, Choosing a Divergence Loss Function – Dropout – Batch Normalization
Module 4: Convolution Neural Network
Convolutional Neural Networks (CNN) - Weights as Templates – Translation Invariance Training with shared parameters – Arriving at the convolutional model, Mathematical details of CNN, Alexnet – Inception – VGG - Transfer Learning
Module 5
Recurrent Neural Networks (RNNs),Modeling sequences - Back propagation through time-Bidirectional RNNs, Exploding/vanishing gradients - Long Short-Term Memory Units (LSTM)
Ian Goodfellow, Yoshua Bengio, and Aaron Courville: Deep learning: The MIT Press, 2016.
Nielsen, Michael A. Neural networks and deep learning. Vol. 25. San Francisco, CA, USA: Determination press, 2015.
Haykin, Simon. Neural networks and learning machines, 3/E. Pearson Education India, 2009.
Bishop, Christopher M., and Nasser M. Nasrabadi. Pattern recognition and machine learning. Vol. 4. No. 4. New York: springer, 2006.
Review Paper: 10 marks
Assignment 1: 5 marks
Assignment 2: 5 marks
Quiz: 5 marks
Internal Examination: 15 marks
Course Plan, Course Coverage and Attendance
Rubrics for Assessment of Review Paper Folder for Uploading Review Paper
TBA
General Instructions
You can form groups of up to 6 students for completing this assignment.
Each batch has to select one gate from Question 1 and one gate from Question 2.
Code for Question 1, along with sample output has to to uploaded here. Answer for Question 2 has to be submitted in class.
Assigned on: 17.04.2023
Due date: 26.04.2023
Maximum points: 100
Penalty for late submission: 25% per day.
Question 1: Implement a 2-input perceptron in any of the scientific computing languages of your choice. Using perceptron learning algorithm, learn the the weights and bias of the separating line so that the perceptron behaves as:
AND gate
OR gate
NAND gate
NOR gate
Question 2: One of the limitations of a perceptron is that it can only learn to discriminate linearly separable data but this limitation can be overcome by combining two or more perceptrons. Explain how you can implement a system using multiple perceptrons to behave as a:
2-input XOR gate
2-input XNOR gate.
General Instructions
Assigned on: 13.09.2023
Due date: 18.09.2023
Maximum points: 100
Answer the Bonus Question for extra points (upto 50 points).
Question 1: Given a basic RNN with a single hidden layer consisting of 50 recurrent neurons and an input sequence that lasts for 8 time steps with each input vector being 20-dimensional, how many parameters are necessary for the training of this neural network? [CO2]
Question 2: How is the training process of a Recurrent Neural Network (RNN) facilitated by propagating errors backwards through time? {CO2]
Question 3: How would you characterize a neural network design that interprets sequences from both starting and ending points? {CO2]
Bonus Question: Imagine being in a grand library, surrounded by ancient tomes and manuscripts. You come across a book titled "Legends of the Descent" that narrates the tales of four legendary navigators: Old Sage Gradient, Momentum Max, Adagrad Ada, and RMSProp Rosie. Each has a unique method for descending the mysterious "Valley of Losses." Your curiosity gets the better of you, and you turn to the librarian and ask, "I'm familiar with Old Sage Gradient, who uses the immediate terrain's feel beneath his feet to decide his next move. But what about the others? How do the methods of Momentum Max with his momentum-packed boots, Adagrad Ada with her history-etched map, and RMSProp Rosie with her fusion crystal ball stand out and possibly enhance the age-old descent technique the Old Sage uses?" [CO1]