CO1: Describe and analyze basics of neural networks and apply the learning rules for deep neural networks.
CO2: Design, implement and apply Deep Neural Networks using CNN and RNN for object detection, image segmentation and text related problems
CO3: Describe mathematical, statistical and computational challenges of building stable representations for high-dimensional data, such as images and text using case studies.
Course Content
Neural Networks: Introduction to Human and Artificial Intelligence, History of AI, Forms of learning,- Supervised and unsupervised learning, Perceptron Learning rule, Bio-inspired learning, Artificial Neural Networks, Backpropagation, Multi-layer Perceptron model, Activation Functions Loss functions, Optimization, Training Neural Networks - gradient descent, stochastic gradient descent, momentum, weight initialization, batch normalization, hyper parameter optimization, parameter updates, model ensembles.
Image Classification using CNNs: Convolutional Neural Networks: convolution layer, pooling layer, fully connected layer, Conv Net, Case study of ImageNet challenge: LeNet, AlexNet, VGG, GoogLeNet, ResNet, Inception Net, Efficient Net etc. Regularization Techniques, Data Augmentation: zooming, rotation, cropping, blurring, noise addition, self-supervision techniques, semi-supervised and weakly supervised learning, adversarial training Transfer Learning, freezing the input layers, fine tuning output layers.
Deep learning for Segmentation and Object detection: Image Localization, Image segmentation, masks, Image segmentation architectures: Unet, VNet, UNet++, Object Detection – Region Proposal Networks, Objection architectures RCNN, Fast and Faster RCNNs, Mask RCNN, YOLO, BiFPN layers, Centre Net, EfficientDet , Case study: RoI cropping in CT images
RNNs and Transformers: Sequential models, Recurrent Neural Networks, Long Short-Term Memory, Gated Recurrent Units, Backpropagation Through Time (BPTT), Transformer Networks,: Introduction to Attention Mechanism - Queries, Keys, and Values, Multi-Head Attention, Self-supervision techniques, Identifying missing words in a paragraph, text summarization.
Hardware and Software requirements for Deep Learning - FPGA, ASIC, GPUs, GPU architectures – Pascal, Volta, Turing & Ampere, Data Parallelism in GPU, Kernels, TPUs, Frameworks for Deep Learning - PyTorch, TensorFlow.
Reference:
Y. Bengio, I. Goodfellow and A. Courville, Deep Learning, MIT Press, 2016.
Nielsen, Michael A. Neural networks and deep learning. Vol. 25. San Francisco, CA, USA: Determination press, 2015.
Bishop, C., M., Pattern Recognition and Machine Learning, Springer , 2006.
Francois Chollet, Deep Learning with Python, Manning Publications, 2017.
Prince, Simon JD. Computer vision: models, learning, and inference. Cambridge University Press, 2012.
Stuart Jonathan Russell, Peter Norvig, Artificial intelligence a modern approach, 4th Edn, Pearson Education, Inc, 2016