Meta-Learning 2nd Ed

Spring 2021

Instructor:

Dr. Gautam Shroff

(TCS Research)

Credits: 1 /2/3 depending on Institute

Link to 1st Ed of Course

About the Course: 'Meta-Learning' or 'learning to learn' are machine-learning/deep-learning techniques which, by experiencing many different learning tasks, can adapt to new tasks/domains/environments, either very "rapidly", i.e., requiring very little new data/experience, or in a manner robust to distributional shifts, such as when users change their behaviour, as has been experienced across the board due to the pandemic. Both scenarios are also closely related, since distributional shifts also imply the relative scarcity of 'new' data, with practical considerations making it imperative to rapidly adapt to every 'new normal'. Meta-learning is also closely related to two long-desired goals of AI in general, viz., continual learning (without forgetting) and learning higher-level (and ideally causal) representations that allow for better generalisation, e.g. to deal with non-stationarity or for longer duration forecasting etc.

In the light of the above context, the course will aim to (i) cover the basic techniques for meta-learning (ii) outline and model selected industry applications in a meta-learning setting and (iii) highlight the connections with closely related long-term AI-research goals.

Lectures (live but online) will be as follows:

Basics of Meta-learning: Metric/Model/Optimisation-based
Meta-Learning for Transfer, Continual learning and Reinforcement Learning
Learning from almost No Data: Meta-interpretive Program Synthesis
Selected applications of Meta-learning in IoT, Optimisation, Healthcare and Advertising
Causality, Disentangled Representations and Causal Reinforcement Learning

Pre-requisite: Python experience, and any course on AI, Data-science, Machine-learning or Deep-learning*; additionally, Lectures 1,2,[5]** and 6B (i.e., from 00:43:40 in the L6 video) of Yann Lecun's Deep Learning course at NYU https://atcold.github.io/pytorch-Deep-Learning/ are mandatory for IIITD and BITS Goa students, and optional for IITD students especially those who have no experience with deep learning or PyTorch.

(* equivalent certifications from reputed universities via online platforms, e.g. Coursera, EdX, Stanford Online, are also fine.)

(** lecture 5 can be done last, i.e., independently of 6B; 2 tutorials for discussion of these lectures will be announced and held in Jan/Feb)

Evaluation: (with differences based on institute):

(i) 3 MCQ quizes for weeks 2-5 for the Meta-learning lectures from March 16-April-30, i.e., in-class, timed for about 15 min each. These will be based on the previous weeks' (plural) lectures' content and assigned reading(s)/hw(s), and are mandatory for all registered students from any of the 3 institutes. See readings for more details.

(ii) Final 1 hour exam (again MCQ, live with multiple questions, individually timed); based on concepts and assigned readings/hws; mandatory for all registered students from any of the 3 institutes.

(iiI) Quiz on deep-learning refresher (Lecun's course above), conducted in-class towards end-Feb, mandatory for BITS and IIITD students.

(iv) Project (individually done) to implement one paper (choices will be given but each student gets a unique paper) as an "educative" .ipynb notebook properly documented with analysis, on a dataset for which results are not already included in the paper. This is mandatory for BITS students.

Time-frame: March 16th (Tue) to April 20 (Tue)

TIMINGS: Tuesdays and Fridays 5:30-7:00 pm. Online: Zoom meeting invites will be shared on email-groups.

Link to lab notebooks

Google Sites

Report abuse

IIT Delhi CSE: COV878 Special Topics in Machine Learning (Deep Learning) 1 credit

IIIT Delhi CSE: CSE663A Meta-Learning 2 credits

BITS Goa CS: BITS G513 (Study in Advanced Topics) part of 4/5-credit course

Meta-Learning 2nd Ed

Spring 2021

Instructor:

Dr. Gautam Shroff

(TCS Research)

In the light of the above context, the course will aim to (i) cover the basic techniques for meta-learning (ii) outline and model selected industry applications in a meta-learning setting and (iii) highlight the connections with closely related long-term AI-research goals.

Lectures (live but online) will be as follows:

Basics of Meta-learning: Metric/Model/Optimisation-based

Meta-Learning for Transfer, Continual learning and Reinforcement Learning

Learning from almost No Data: Meta-interpretive Program Synthesis

Selected applications of Meta-learning in IoT, Optimisation, Healthcare and Advertising

Causality, Disentangled Representations and Causal Reinforcement Learning

Evaluation: (with differences based on institute):

Time-frame: March 16th (Tue) to April 20 (Tue)

TIMINGS: Tuesdays and Fridays 5:30-7:00 pm. Online: Zoom meeting invites will be shared on email-groups.

Link to lab notebooks