IIT Delhi CSE: COV878 Special Topics in Machine Learning (Deep Learning) 1 credit
IIIT Delhi CSE: CSE663A Meta-Learning 2 credits
BITS Goa CS: BITS G513 (Study in Advanced Topics) part of 4/5-credit course
Meta-Learning 2nd Ed
Instructor:
Dr. Gautam Shroff
(TCS Research)
Credits: 1 /2/3 depending on Institute
About the Course: 'Meta-Learning' or 'learning to learn' are machine-learning/deep-learning techniques which, by experiencing many different learning tasks, can adapt to new tasks/domains/environments, either very "rapidly", i.e., requiring very little new data/experience, or in a manner robust to distributional shifts, such as when users change their behaviour, as has been experienced across the board due to the pandemic. Both scenarios are also closely related, since distributional shifts also imply the relative scarcity of 'new' data, with practical considerations making it imperative to rapidly adapt to every 'new normal'. Meta-learning is also closely related to two long-desired goals of AI in general, viz., continual learning (without forgetting) and learning higher-level (and ideally causal) representations that allow for better generalisation, e.g. to deal with non-stationarity or for longer duration forecasting etc.
In the light of the above context, the course will aim to (i) cover the basic techniques for meta-learning (ii) outline and model selected industry applications in a meta-learning setting and (iii) highlight the connections with closely related long-term AI-research goals.
Lectures (live but online) will be as follows:
- Basics of Meta-learning: Metric/Model/Optimisation-based
- Meta-Learning for Transfer, Continual learning and Reinforcement Learning
- Learning from almost No Data: Meta-interpretive Program Synthesis
- Selected applications of Meta-learning in IoT, Optimisation, Healthcare and Advertising
- Causality, Disentangled Representations and Causal Reinforcement Learning
Pre-requisite: Python experience, and any course on AI, Data-science, Machine-learning or Deep-learning*; additionally, Lectures 1,2,[5]** and 6B (i.e., from 00:43:40 in the L6 video) of Yann Lecun's Deep Learning course at NYU https://atcold.github.io/pytorch-Deep-Learning/ are mandatory for IIITD and BITS Goa students, and optional for IITD students especially those who have no experience with deep learning or PyTorch.
(* equivalent certifications from reputed universities via online platforms, e.g. Coursera, EdX, Stanford Online, are also fine.)
(** lecture 5 can be done last, i.e., independently of 6B; 2 tutorials for discussion of these lectures will be announced and held in Jan/Feb)
Evaluation: (with differences based on institute):
(i) 3 MCQ quizes for weeks 2-5 for the Meta-learning lectures from March 16-April-30, i.e., in-class, timed for about 15 min each. These will be based on the previous weeks' (plural) lectures' content and assigned reading(s)/hw(s), and are mandatory for all registered students from any of the 3 institutes. See readings for more details.
(ii) Final 1 hour exam (again MCQ, live with multiple questions, individually timed); based on concepts and assigned readings/hws; mandatory for all registered students from any of the 3 institutes.
(iiI) Quiz on deep-learning refresher (Lecun's course above), conducted in-class towards end-Feb, mandatory for BITS and IIITD students.
(iv) Project (individually done) to implement one paper (choices will be given but each student gets a unique paper) as an "educative" .ipynb notebook properly documented with analysis, on a dataset for which results are not already included in the paper. This is mandatory for BITS students.