Description

Deep Learning has revolutionized both the scientific and industrial worlds. It is one of the most highly sought-after skills in AI with hundreds of thousand job openings each year.

In this course, you will learn the foundations of Deep Learning, understand how to build neural networks using Pytorch. You will learn about Convolutional networks, RNNs, Transformers architectures. You will get to use those architectures in popular applications of Deep Learning such as Transfer Learning, Self-Supervised Learning, Generative Models, Graph Neural Networks, and so on.

Upon completion of the course, students will be able to:

• Understand the fundamental concepts and techniques of supervised and unsupervised learning in Deep Learning.

• Design, implement, and train shallow and deep neural networks.

• Apply advanced techniques such as CNN, Resnets, and Transformers.

• Experiment with self-supervised learning and meta-learning approaches.

• Analyze and implement geometric and equivariant neural network models.

• Evaluate the performance of deep learning models and apply regularization and compression techniques.

• Understand the challenges and solutions related to noise robustness in deep learning models.

• Apply the knowledge gained in hands-on projects using tools like Pytorch and HuggingFace.

Syllabus

• Introduction to Deep Learning and course administration.

•  Supervised Learning.

•  Shallow Neural Networks.

•  Deep Neural Networks.

•  Model Training: Loss function.

•  Model Training: Fitting.

•  Model Training: Backpropagation.

•  Measuring Performance.

•  Regularization.

•  CNN & Resnets.

•  Self-Supervised Learning.

•  Transformers.

•  Retrieval Augmented Generation.

•  Generative models: Unsupervised Learning and Generative Adversarial Networks.

• Graph Neural Networks.

• Geometric Deep Learning.

• Meta-Learning.

• Deep Learning Model Compression.

• Noise Robustness of Deep Learning Models.

Course Material

Understanding Deep Learning - by Simon J.D. Prince - To be published by MIT Press Dec 5th 2023. Available at https://udlbook.github.io/udlbook/ Plus articles shared by the teacher

Exam

The student's knowledge will be assessed through two main components:

• Project: Throughout the course, a project will be assigned where students must implement and test deep learning algorithms and techniques. This project will account for about 30% of the final grade (i.e., up to 10 points over 30).

• Written Exam: Students will be assessed on their understanding of the course's key concepts and their ability to discuss and apply them. The written exam will account for 70% of the final grade (i.e., up to 21 points over 30)

The final grade will combine these two components, and if the sup is 31 the exam will be considered passed "cum laude".