PREparatory classes, 2.5 ECTS each. Mandatory for all M1 [AI] students, except PRE3 and PRE4.
Prerequisites for M2 students who want to follow M2 [AI] classes.
If you cannot take them, study on your own online CRASH COURSES.

• • PRE1: APPLIED STATISTICS 

 Instructor: Stéphane Rivaud  

This course introduces the necessary formalism to solve computational problems with statistical reasoning. The goal is to provide students with the necessary skills to employ statistical modeling in a computational framework to tackle real world problems. 

By the end, students should be able to:

•  Give a statistical description of a dataset.

•  Leverage probabilistic modeling to make statistical inference.

•  Analyze the mathematical properties of a statistical estimator.

•  Employ bootstrapping.

•  Perform hypothesis testing.

• • PRE2: MATHEMATICS FOR DATA SCIENCE 

Instructor: Eva Feillet 

This class aims at teaching/reminding mathematical basis useful in data science, such as: 

•  vector spaces, linear transformations, 

•  matrices, linear systems, 

•  norms, orthogonality, 

•  eigenvalues, singular value decomposition, 

•  tensors (notions), multivariable calculus

Old content: online-class, website- Mathématiques pour les sciences des données 

• • PRE3: RELATIONAL DATABASES  

Instructor: Fatiha Saïs

This course aims enabling the students to learn the fundamentals of Relational Database Management Systems (DBMS) and apply these concepts in practice. 

•  Basics of relational databases

•  Design theory for relational databases

•  Relational algebra and SQL

•  High-level database models

Old content: ecampus page (from last year) - Bases de données relationnelles, SQL

• • PRE4: SCIENTIFIC PROGRAMMING This course is not offered this year.

Tronc commun (TC) “classic classes", 2,5 ECTS each. 

To get a consistent path of study with adequate prerequisites, among optional classes, always take either green (ML path) or brown (NLP path) classes (or both).

• • TC0: Foundational Principles of Machine Learning 

Instructor:  François Landes

This course is algorithms-oriented, i.e. we will sketch the great principles of ML at first, and then focus on how algorithms work in practice, including all necessary mathematical aspects. They are the basic building blocks of more advanced algorithms.

•  Gradient Descent, Linear Regression from scratch

•  Classification with a single layer Perceptron, from scratch. Geometrical interpretation, a word on SGD/mini-batch learning. Discussion on the choice of the loss function, or activation functions. OVR multi-class scheme (quickly)

•  overfitting, train/validation/test split, K-fold CV, regularization: in general, L2, L1.

•  MAP, Bayesian interpretation of Ridge Regression or Lasso one.

•  feature maps ("Kernel trick"), PCA (from scratch, seen as variance maximization), PCA as pre-processing (dimensional reduction)

•  Kernels, Kernelized perceptron, SVM in the separable case (some details omitted). Exercise: Lasso regularization from scratch (pseudo-gradients quickly introduced)

 (FPML, organization, all material here) - [ecampus: TBA].- Introduction to Machine Learning -- with PRE1 and PRE2 as prerequisites, and PRE4 is strongly recommended. Note that PRE1 and PRE2 are mandatory, i.e., you must attend them to be allowed to follow TC0/FPML, except if you can argue that you are already very fluent in statistics (PRE1) and linear algebra (PRE2). This class (or an equivalent) is a prerequisit for almost all other [AI] classes. 

• • TC1: Machine Learning Algorithms 

Instructors: Faïcel Chamroukhi, Sylvain Chevallier

This course aims at mastering the core concept of algorithmic design in ML, from an optimization or a probabilitic point-of-view, using supervised and unsupervised algorithms

•  Regression/classification seen in optimization and probabilistic frameworks, implication on batch and stochastic gradient descent

• Learning theory and Vapnick-Charvonenkis dimension

• Evaluating performances of ML algorithms in different contexts (imbalanced, small-sized, etc) 

• Probabilistic framework for machine learning: Discriminative vs Generative learning, Empirical Risk Minimization, Risk Decomposition,  Bias-Variance Tradeoff; Maximum Likelihood Estimation (MLE),  MLE and OLS in regression, MLE and IRLS in softmax classification   

•  Unsupervised Learning and Clustering: K-means, Mixture Models, EM algorithms,..)

•  Unsupervised Learning and Dimensionality reduction: PCA, Probabilistic PCA & EM, ICA,..

 [external site] -- with TC0 as prerequisite. This class is a prerequisite for OPT4 (DL).

• • TC2: OPTIMIZATION 

Instructor: Faïcel Chamroukhi

This Optimization for Machine Learning course aims at studying the mathematical and computational constructions and properties of key optimization algorithms in different situations (continuous/discrete, constrained/unconstrained, convex/non-convex, deterministic, and stochastic problems) with use case illustrations in machine learning.

•  Introductions and background (convexity, differentiability, optimality conditions, convergence rates …)

•  Continuous optimization (first order methods: Gradient methods, linear search, Acceleration)

•  Continuous optimization (second order methods: Newton methods including Quasi-Newton, secant, IRLS)

•  Constrained optimization (Equality and Inequality constraints, Duality/Lagrangian, KKT optimality conditions, Linear Programming, GD for a constrained problem)

•  Non-convex, stochastic optimization (The EM Algorithm, Stochastic Gradient, Stochastic EM)

             website 2024/2025 - Optimisation, descente de gradient, etc. -- with PRE2 as prerequisite

• • TC3: INFORMATION RETRIEVAL

Instructor:  Kim Gerdes

This course gives a basic introduction to Information Retrieval

•  Introduction to Information Retrieval: Introduction to key terms and domains; tutorial on new textual dataset indexing and basic counting techniques.

•  Handling Large Datasets: Exploration of big datasets; binary evaluation methods; introduction to TF-IDF.

•  Improving Retrieval Methods: Introduction to sparse embeddings; overview of BM25 and Sense2Vec.

•  Advanced Embedding Techniques: Study of dense embeddings; use case with a patent dataset and its citations; introduction to doc2vec and sentenceBERT.

•  Challenge Presentation: Presentation of class challenge related to information retrieval concepts.

•  Project Work and Presentation: Collaborative project work and discussions; final project presentations.

 with PRE1, 2, and OPT17 "Hands-on NLP" as prerequisite.

Formerly all OPTional classes, though some in bold, are now mandatory for the [AI] track :-) 2.5 ECTS each. 

• • OPT4: DEEP LEARNING 

Instructor: Thomas Gerald

The aim of this course is to introduce the Deep-Learning framework. It will cover fundamental models such as the multilayer perceptron through to the most recent deep learning architectures (CNN, VAE,...). In addition, the course will cover the different approaches to train these neural networks, with lectures dedicated to backpropagation algorithms and optimization methods based on gradient descent.

•  Introduction to Neural-Networks and the MLP model 

•  MLP and Gradient Descent algorithm

•  Backpropagation algorithm and optimization methods

•  Create you neural network with pytorch

•  Neural Networks architectures (CNN, AE, ...)

•  Generative approaches (VAE,GAN, Denoising Diffusion models)

 With TC1 as prerequisite. Highly recommended for all, mandatory for all [AI] students.

• • OPT8: HISTORY OF AI

Instructor: Kim Gerdes

The "History of AI" course aims to give students an overview of the development of artificial intelligence from its beginnings to its current state. Students will learn about key events, major ideas, and ethical issues in AI, helping them understand how today's AI tools came to be and their impact on society.

•   Introduction to AI and Its History

• Overview of the course, discussion on what AI is, its current issues, and a brief look at the history of AI development.

• Philosophical Foundations of AI

• Introduction to key philosophical ideas in AI, including the Turing Test, John Searle's Chinese Room argument, and Leibniz's thoughts on computation.

• Major Periods in AI Development

• Overview of important times in AI history, focusing on the growth and decline of interest and funding in AI research (known as AI Summer and Winter).

• Development of AI Techniques

•  Exploration of different methods used in AI, the link between linguistics and AI, and how gaming technology has influenced deep learning

• Machine Creativity and Future Predictions

• Discussion on how AI can create art, music, and literature, and what the future might hold for AI, including the idea of the singularity.

• Ethics and Future Challenges in AI

• Examination of ethical issues, political considerations, and future challenges in AI, focusing on the responsibilities of AI researchers and the impact on society

 [ecampus]

• • OPT9: Hands-on Machine Learning 

Instructor: Marc Evrard

A practical oriented class, where students apply ML techniques to simple illustrative examples and then to tackle competitive challenges. It will start with an introduction to present (refresh) the ML landscape. Classes will then be articulated to successively focus on the major concepts of practical ML. Outline:

•  Introduction/refresher on ML

•  Working with real data

•  Discover and visualize the data to gain insights

•  Prepare the data for processing

•  Select and train models

• Fine-tune models

[external site] -- With PRE1 and PRE2 as prerequisite.

• • OPT13: INFORMATION THEORY

Instructor:  Guillaume Charpiat

TD: Styliani Douka

Information theory provides a theoretical ground for ML in general. 

The problems we aim at solving:

•  how to complete a sequence?  1 2 3 5 7 11 13 17 ... ?? Why is 19 more "probable"? how to justify it?

•  how to decide between two models for given data?

•  how to set a ML problem? which criterion to optimize? how to measure a solution's performance?

          and this will lead us to the following problems:

• how to quantify information?

•  is there a "natural" distribution over numbers? over models? 

• how to compress (losslessly) data? are there bounds?

In more details: 

We study the tools from Information Theory that are useful in Machine Learning. We introduce the concept of entropy, leading to a distance measure between distributions (Kullback-Leibler divergence). We then study the equivalence between compression, prediction and generation. In a third part, we get a glimpse of information geometry (Fisher metric). Last, we formalize the preference for simpler models through Kolmogorov complexity.

Chapter 1 : Entropy

Chapter 2 : Compression/Prediction/Generation equivalence

Chapter 3 : Fisher information 

Chapter 4 : Kolmogorov complexity

Course website: https://www.lri.fr/~gcharpia/informationtheory/

 (website from a previous similar class). Théorie de l'information -- With PRE1 as prerequisite

• • OPT 15: Fairness in AI

Instructor: Adrian Popescu

This course discusses theoretical and practical aspects of fairness in AI. The objectives are to: 

•  Highilight the importance of building fair AI systems and analyze the legal, ethical, social and technical challenges of this process.

•  Present the main types of biases appearing in AI systems and describe the existing approaches to mitigate them. 

•  Illustrate the practical impact of (un)fairness through impactful applications such as: personal data processing, news analysis, and face recognition.

•  Develop you critical skills with hands-on analysis of recent research paper and programming activities. 

• • OPT 16: Data camp

Instructor:  Thomas Moreau

This course aims to learn the practical tools for datascience and how to frame and solve datascience problems.

• Data wrangling

•  The scikit-learn API and Missing values

•  Metrics and unbalanced data

•  Dealing with complex data

•  Ensemble methods and hyperparameter optimization

Full syllabus: https://egrcc.github.io/docs/math/all-of-statistics.pdf r

 Old content:  (Website from previous years) [ecampus].: Create a challenge (that other students will solve as a TER project). Team work in teams of 5-6 people. -- With TC0 and PRE4 (or equivalent) as prerequisite.

• • OPT 17: Hands-on Natural Language Processing

Instructors:  Kim Gerdes, Nona Naderi

•  Introduction to NLP

•  Basic concepts in NLP, tokenization, lemmatization, POS tagging,...

•  Lexical semantics, word sense disambiguation

•  Syntax and interpretations

•  Parsing

• OPT 16: Creation of a challenge in Artificial Intelligence

Instructor: Lisheng Sun-Hosoya

This course is designed to bridge theory (lectures) and practice (TPs) by guiding students through the end-to-end creation and resolution of an AI challenge, using pre-formatted real datasets and the open-source platform Codabench.

1: Designing AI Challenges

2: Evaluation and Metrics 

3: Baselines: Core ML algorithms

4: Baselines: Image Classification

5: Optimizing ML Solutions

6: Interpreting Results & Presenting Impact

7: Project presentations

full syllabus: https://docs.google.com/document/d/1c81yUmGwvNgNcB-z3hVvPZwrxLCDtiW3wAK1A14hfhY/edit?tab=t.0

• • Summer school (école thématique)

  • Internship (stage) (Coordination: Nona Naderi)

  • TER: Travail d'Etude et de Recherche (small internship). Personal work supervised by a member of the master's teaching team or in s university research lab. This work can take the form of a state of the art on a given scientific subject and / or the implementation of state algorithms for application on a given problem. This work is normally not remunerated. However, If an internship is longer than 2 months it must be remunerated. (Coordination: Nona Naderi)

• • Languages: French or English for non-native speakers.

  • I&E: Innovation and Entrepreneurship.

  • FVE: Research and Development training, mutualised with avec MIAGE d'Orsay. Formation à la Vie en Entreprise

  • Teacher responsible for I&E basics/BDlabs (M1 EIT) is: Guillaume DION

Core Curriculum (Tronc Commun, TC): Mandatory foundational courses for all AI students

T5B: Probabilistic Generative Models

Instructor: Bertrand Delezoide – bertrand.delezoide@gmail.com

Keywords: Notions of statistics and AI

Prerequisites: PRE1 and PRE2, or equivalent courses

• Foundations of Generative Models – definitions, probability basics, explicit vs. implicit models, first applications
  
  

• Graphical Models & HMMs – Bayesian and Markov networks, exact and approximate inference, sequential models
  
  

• Neural Generative Models (Images) – VAEs, GANs, and diffusion models for image synthesis
  
  

• Text Generation with Transformers – architecture, attention, pre-training (BERT, GPT)
  
  

• Optimization & Advanced LLM Techniques – efficient fine-tuning with methods such as LoRA
  
  

• Emerging Models – liquid neural networks, hierarchical recurrent models, state-space and quantum-inspired approaches
  
  

T5C: Signal Processing

Instructor: Matthieu Kowalski – matthieu.kowalski@universite-paris-saclay.fr, matthieu.kowalski@inria.fr

Keywords: Spectral analysis of time signals and images, convolution and filtering, time-frequency and wavelet analysis, denoising, sparse coding, inverse problems

Course page: http://hebergement.universite-paris-saclay.fr/mkowalski/SigPro/

Prerequisites: PRE2 or equivalent courses

• Introduction & Fourier Analysis
  
  

• Filtering Methods
  
  

• Random Signals and Stochastic Processes
  
  

• Time-Frequency Analysis
  
  

• Time-Scale (Wavelet) Analysis
  
  

• Introduction to Linear Inverse Problems
  
  

T5A: Frugal AI

Instructor: Adrian Popescu – adrian.popescu@cea.fr

Keywords: pretraining, large models, transfer learning, data scarcity

Prerequisites: notions of AI, deep learning, and Python programming

• Frugality in AI – overview of constraints on data, model parameters, training, inference, and memory; relation to large-scale models
  
  

• Label Frugality – supervised, weakly-supervised, semi-supervised, and self-supervised learning
  
  

• Data Frugality – transfer, few-shot, zero-shot, and continual learning
  
  

• Frugal vs. Large Deep Models – comparative analysis and practical applications
  
  

• Labs: self-supervised learning, few-shot learning, and continual learning
  
  

T5D: Image Mining

Instructor: Antoine Manzanera – antoine.manzanera@ensta-paris.fr

Keywords: image representation, mining and retrieval

Course page: https://perso.ensta-paris.fr/~manzaner/Cours/Masters_ParisSaclay/Image_Mining/

Prerequisites: Linear algebra, differential calculus, probability, statistics, signal processing

• Image Models and Representations
  
  

• Feature Extraction Techniques
  
  

• Image Classification
  
  

• Visual Tracking
  
  

• Remote Sensing Applications
  
  

• Deep Learning for Visual Data
  
  

T6A: Deep Learning for NLP

Instructor: Nona Naderi – nona.naderi@universite-paris-saclay.fr

Keywords: machine learning, statistics, probability theory, Python

Prerequisites: basic ML and deep learning background

• Introduction to NLP and Language Models
  
  

• Convolutional Neural Networks for Text
  
  

• Recurrent Neural Networks
  
  

• Attention Mechanisms
  
  

• Word Representations and Transformers
  
  

• Generative Language Models
  
  

T6B: Reinforcement Learning

Instructor: Lionel Mathelin – lionel.mathelin@lisn.upsaclay.fr

Keywords: machine learning, statistics, probability theory, Python

Prerequisites: solid understanding of ML fundamentals

• Introduction to Reinforcement Learning

• Markov Decision Processes (MDPs)

• Planning by Dynamic Programming

• Model-Free Prediction and Control

• Value Function Approximation

• Policy Gradient Methods
  
  

T6C: Speech and Language Processing

Instructors: Kim Gerdes, Marc Evrard – gerdes@lisn.fr

Keywords: ongoing NLP research at LISN, linguistics and NLP integration

Prerequisites: knowledge from M1 Hands-On NLP (tokens, embeddings, annotated text data, notebooks)

• From Linguistics to NLP

• Treebanks and Oral Syntax

• Leveraging Geolocated Speech

• Speech-Audio Processing and ASR

• Emotion Detection in Speech

• Speech Interaction and Dialogue Systems
  
  

T6D: Computer Vision

Instructor: Eva Feillet – eva.feillet@protonmail.com

Keywords: linear algebra, signal processing, differential calculus, neural networks

Prerequisites: fundamentals of ML, signal processing, and calculus

• Challenges of Visual Data and Color Spaces
  
  

• Image Processing Fundamentals
  
  

• Geometry and Segmentation
  
  

• Convolutional Neural Networks (CNNs)
  
  

• Transfer Learning and Visual Representation Learning
  
  

• Transformers for Vision
  
  

T7A: Learning Theory and Advanced Machine Learning

Instructor: Antoine Cornuéjols – antoine.cornuejols@agroparistech.fr

Course page: https://antoinecornuejols.github.io/teaching/Master-AIC/M2-AIC-advanced-ML.html

Keywords: statistical learning theory, ensemble methods, semi-supervised learning, domain adaptation, continual learning

• Statistical Learning Theory and the Problem of Induction
  
  

• Ensemble Methods and Collaborative Learning Agents
  
  

• Learning from Imbalanced and Non-Stationary Data
  
  

• Semi-Supervised and Domain Adaptation Techniques
  
  

• Continual Learning and Knowledge Transfer
  
  

• Student Projects: critical appraisal and replication of recent research papers
  
  

T7B: Advanced Optimization and Automated Machine Learning

Instructor: Solal Nathan – solal.nathan@universite-paris-saclay.fr, solal.nathan@inria.fr

Keywords: optimization, black-box optimization, Bayesian optimization, neural architecture search, hyperparameter optimization, meta-learning, AutoML

Prerequisites: solid foundations in ML and optimization

• Introduction to Optimization for ML
  
  

• Unconstrained and Constrained Continuous Optimization
  
  

• Black-Box and Bayesian Optimization
  
  

• Hyperparameter Optimization
  
  

• Neural Architecture Search (NAS)
  
  

• Learning to Optimize and Meta-Learning
  
  

T7C: Text Mining and Chatbots

Instructors: Cyril Grouin, Thomas Gérald – cyril.grouin@limsi.fr, thomas.gerald@universite-paris-saclay.fr

Course page: https://perso.limsi.fr/grouin/m2aic/

Keywords: text processing, word representations, neural networks for texts, dialogue systems

Prerequisites: NLP, Python, and PyTorch

• Introduction to Text Mining
  
  

• Text Mining in Open and Medical Domains
  
  

• Semantics and Word Embeddings
  
  

• Chatbots and Evaluation Methods
  
  

• Practical Labs: chatbot development and evaluation
  
  

T7D: Scientific Machine Learning

Instructor: Alena Shilova – alena.shilova@inria.fr

Keywords: Graph Neural Networks (GNNs), NeuralODEs, Physics-Informed Neural Networks (PINNs), Geometric Deep Learning, Transformers

Prerequisites: background in deep learning and applied mathematics

• Introduction to Scientific Machine Learning
  
  

• Applications in Physics, Chemistry, and Biology
  
  

• Graph Neural Networks and Geometric Deep Learning
  
  

• NeuralODEs and PINNs
  
  

• Steerable and Equivariant Neural Networks
  
  

• Attention Mechanisms across Architectures (GAT, EquiFormer, ViT)
  
  

• Communitation

• I&E: Innovation and Entrepreneurship.

• Conferences

• Teacher responsible for I&E Study (M2 EIT ) is: Alvaro PINA STRANGER

5 to 6 month internship in a research lab or a company (Coordination: Marc Evrard and Thomas Gerald).