19CSE456OddSem2023-24NNDL

19CSE456: Neural Networks and Deep Learning

(Odd Semester 2023-2024)

(V SEM-BTech CSE-PE1)

Instructor: Dr. Mansi Sharma

Lectures: Thrusday (Slot 5), Friday (Slot 4)

Lab Sessions: Monday (Slot 10-11)

Room No. - AB3 GF C103

Course Description

Prerequisite

19MAT111 Multivariable Calculus

19MAT112 Linear Algebra

19MAT205 Probability and Random Processes

19CSE305 Machine Learning

Syllabus

McCulloch Pitts Neuron, Thresholding Logic, Perceptrons, Perceptron Learning Algorithm and Convergence, Multilayer Perceptrons (MLPs), Representation Power of MLPs

Sigmoid Neurons, Gradient Descent, Feedforward Neural Networks, Representation Power of Feedforward Neural Networks, Backpropagation

Gradient Descent (GD), Momentum Based GD, Nesterov Accelerated GD, Stochastic GD, AdaGrad, RMSProp, Adam

Eigen Values, Eigen Vectors, Eigen Value Decomposition, Principal Component Analysis, Singular Value Decomposition

Autoencoders and relation to PCA, Regularization in autoencoders, Denoising autoencoders, Sparse autoencoders, Contractive autoencoders

Regularization: Bias Variance Tradeoff, l2 regularization, Early stopping, Dataset augmentation, Parameter sharing and tying, Injecting noise at input, Ensemble methods, Dropout

Greedy Layerwise Pre-training, Better activation functions, Better weight initialization methods, Batch Normalization

Learning Vectorial Representations Of Words

Convolutional Neural Networks, LeNet, AlexNet, ZF-Net, VGGNet, GoogLeNet and ResNet

Sequence Learning Problems, Recurrent Neural Networks, Backpropagation Through Time (BPTT), Vanishing and Exploding Gradients, Truncated BPTT

Long Short Term Memory Cells (LSTMs), Gated Recurrent Units (GRUs)

Encoder Decoder Models, Attention Mechanism

Probabilistic Neural Network, Hopfield Net, Bayesian Network, Markov Network, Boltzman Machine, Restricted Boltzmann Machines, Conditional Random Fields, Belief Propagation, Hidden Markov Model

Evaluation Plan

Continuous Assessment (Theory)

Two Tutorials (2 x 5 = 10 Marks)

Continuous Assessment (Lab)

Three Lab Evaluations

Lab 1 Evaluation 10 Marks

Lab 2 Evaluation 10 Marks

Lab 3 Project Based 20 Marks

Mid Term Exam 20 Marks

End Semester Exam 30 Marks

Tentative Exam Schedule

Tutorial 1 - Aug 24th 2023

Lab 1 Evaluation - Sep 25th 2023

Mid Term Exam - Oct 9th 2023

Tutorial 2 - Oct 27th 2023

Lab 2 Evaluation - Nov 24th 2023

Lab 3/Project Evaluation - Nov 27th 2023

End Semester Exam - Dec 15th 2023

TextBooks/References