PREparatory classes, 2.5 ECTS each. Mandatory for all M1 [AI] students, except PRE3 and PRE4.
Prerequisites for M2 students who want to follow M2 [AI] classes.
If you cannot take them, study on your own online CRASH COURSES.

• • PRE1: APPLIED STATISTICS 

 Instructor: Stéphane Rivaud  

This course introduces the necessary formalism to solve computational problems with statistical reasoning. The goal is to provide students with the necessary skills to employ statistical modeling in a computational framework to tackle real world problems. 

By the end, students should be able to:

•  Give a statistical description of a dataset.

•  Leverage probabilistic modeling to make statistical inference.

•  Analyze the mathematical properties of a statistical estimator.

•  Employ bootstrapping.

•  Perform hypothesis testing.

Old content: online class, website. - Statistiques appliquées

• • PRE2: MATHEMATICS FOR DATA SCIENCE  (details)  

Instructor: Marcella Bonazzoli 

This class aims at teaching/reminding mathematical basis useful in data science, such as: 

•  vector spaces, linear transformations, 

•  matrices, linear systems, 

•  norms, orthogonality, 

•  eigenvalues, singular value decomposition, 

•  tensors (notions), multivariable calculus

Old content: online-class, website- Mathématiques pour les sciences des données 

• • PRE3: RELATIONAL DATABASES  

Instructor: Fatiha Saïs

This course aims enabling the students to learn the fundamentals of Relational Database Management Systems (DBMS) and apply these concepts in practice. 

•  Basics of relational databases

•  Design theory for relational databases

•  Relational algebra and SQL

•  High-level database models

Old content: ecampus page (details from last year) - Bases de données relationnelles, SQL

• • PRE4: SCIENTIFIC PROGRAMMING This course is not offered this year.

Old content:  (syllabus) -online-class, website- Programmation scientifique en Python

Tronc commun (TC) “classic classes", 2,5 ECTS each. 

To get a consistent path of study with adequate prerequisites, among optional classes, always take either green (ML path) or brown (NLP path) classes (or both).

• • TC0: Foundational Principles of Machine Learning 

Instructor:  François Landes

This course is algorithms-oriented, i.e. we will sketch the great principles of ML at first, and then focus on how algorithms work in practice, including all necessary mathematical aspects. They are the basic building blocks of more advanced algorithms.

•  Gradient Descent, Linear Regression from scratch

•  Classification with a single layer Perceptron, from scratch. Geometrical interpretation, a word on SGD/mini-batch learning. Discussion on the choice of the loss function, or activation functions. OVR multi-class scheme (quickly)

•  overfitting, train/validation/test split, K-fold CV, regularization: in general, L2, L1.

•  MAP, Bayesian interpretation of Ridge Regression or Lasso one.

•  feature maps ("Kernel trick"), PCA (from scratch, seen as variance maximization), PCA as pre-processing (dimensional reduction)

•  Kernels, Kernelized perceptron, SVM in the separable case (some details omitted). Exercise: Lasso regularization from scratch (pseudo-gradients quickly introduced)

 (FPML, organization, all material here) - [ecampus: TBA].- Introduction to Machine Learning -- with PRE1 and PRE2 as prerequisites, and PRE4 is strongly recommended. Note that PRE1 and PRE2 are mandatory, i.e., you must attend them to be allowed to follow TC0/FPML, except if you can argue that you are already very fluent in statistics (PRE1) and linear algebra (PRE2). This class (or an equivalent) is a prerequisite for almost all other [AI] classes. 

• • TC1: Machine Learning Algorithms 

Instructors: Faïcel Chamroukhi, Sylvain Chevallier

This course aims at mastering the core concept of algorithmic design in ML, from an optimization or a probabilitic point-of-view, using supervised and unsupervised algorithms

•  Regression/classification seen in optimization and probabilistic frameworks, implication on batch and stochastic gradient descent

• Learning theory and Vapnick-Charvonenkis dimension

• Evaluating performances of ML algorithms in different contexts (imbalanced, small-sized, etc) 

• Probabilistic framework for machine learning: Discriminative vs Generative learning, Empirical Risk Minimization, Risk Decomposition,  Bias-Variance Tradeoff; Maximum Likelihood Estimation (MLE),  MLE and OLS in regression, MLE and IRLS in softmax classification   

•  Unsupervised Learning and Clustering: K-means, Mixture Models, EM algorithms,..)

•  Unsupervised Learning and Dimensionality reduction: PCA, Probabilistic PCA & EM, ICA,..

 [external site][ecampus]. - Algorithmes d'apprentissage -- with TC0 as prerequisite. This class is a prerequisite for OPT4 (DL).

• • TC2: OPTIMIZATION 

Instructor: Faïcel Chamroukhi

This Optimization for Machine Learning course aims at studying the mathematical and computational constructions and properties of key optimization algorithms in different situations (continuous/discrete, constrained/unconstrained, convex/non-convex, deterministic, and stochastic problems) with use case illustrations in machine learning.

•  Introductions and background (convexity, differentiability, optimality conditions, convergence rates …)

•  Continuous optimization (first order methods: Gradient methods, linear search, Acceleration)

•  Continuous optimization (second order methods: Newton methods including Quasi-Newton, secant, IRLS)

•  Constrained optimization (Equality and Inequality constraints, Duality/Lagrangian, KKT optimality conditions, Linear Programming, GD for a constrained problem)

•  Discrete optimization (Branch-and-Bound, Simplex method)

•  Non-convex, stochastic optimization (The EM Algorithm, Stochastic Gradient, Stochastic EM)

             website 2021/2022 - Optimisation, descente de gradient, etc. -- with PRE2 as prerequisite

• • TC3: INFORMATION RETRIEVAL

Instructor:  Kim Gerdes

This course gives a basic introduction to Information Retrieval

•  Introduction to Information Retrieval: Introduction to key terms and domains; tutorial on new textual dataset indexing and basic counting techniques.

•  Handling Large Datasets: Exploration of big datasets; binary evaluation methods; introduction to TF-IDF.

•  Improving Retrieval Methods: Introduction to sparse embeddings; overview of BM25 and Sense2Vec.

•  Advanced Embedding Techniques: Study of dense embeddings; use case with a patent dataset and its citations; introduction to doc2vec and sentenceBERT.

•  Challenge Presentation: Presentation of class challenge related to information retrieval concepts.

•  Project Work and Presentation: Collaborative project work and discussions; final project presentations.

 (website from previous year)-[ecampus].- Recherche et extraction d’information dans les textes  -- with PRE1, 2, and OPT17 "Hands-on NLP" as prerequisite.

• • TC6: Large-Scale Distributed Data Processing (website from last year) -- Algorithmes distribués et bases de données -- with PRE3 as prerequisite (or good knowledge of database knowledge systems, at least of SQL).

Formerly all OPTional classes, though some in bold, are now mandatory for the [AI] track :-) 2.5 ECTS each. 

• • OPT4: DEEP LEARNING 

Instructor: Thomas Gerald

The aim of this course is to introduce the Deep-Learning framework. It will cover fundamental models such as the multilayer perceptron through to the most recent deep learning architectures (CNN, VAE,...). In addition, the course will cover the different approaches to train these neural networks, with lectures dedicated to backpropagation algorithms and optimization methods based on gradient descent.

•  Introduction to Neural-Networks and the MLP model 

•  MLP and Gradient Descent algorithm

•  Backpropagation algorithm and optimization methods

•  Create you neural network with pytorch

•  Neural Networks architectures (CNN, AE, ...)

•  Generative approaches (VAE,GAN, Denoising Diffusion models)

 With TC1 as prerequisite. Highly recommended for all, mandatory for all [AI] students.

• • OPT8: HISTORY OF AI

Instructor: Kim Gerdes

The "History of AI" course aims to give students an overview of the development of artificial intelligence from its beginnings to its current state. Students will learn about key events, major ideas, and ethical issues in AI, helping them understand how today's AI tools came to be and their impact on society.

•   Introduction to AI and Its History

• Overview of the course, discussion on what AI is, its current issues, and a brief look at the history of AI development.

• Philosophical Foundations of AI

• Introduction to key philosophical ideas in AI, including the Turing Test, John Searle's Chinese Room argument, and Leibniz's thoughts on computation.

• Major Periods in AI Development

• Overview of important times in AI history, focusing on the growth and decline of interest and funding in AI research (known as AI Summer and Winter).

• Development of AI Techniques

•  Exploration of different methods used in AI, the link between linguistics and AI, and how gaming technology has influenced deep learning

• Machine Creativity and Future Predictions

• Discussion on how AI can create art, music, and literature, and what the future might hold for AI, including the idea of the singularity.

• Ethics and Future Challenges in AI

• Examination of ethical issues, political considerations, and future challenges in AI, focusing on the responsibilities of AI researchers and the impact on society

 [ecampus]

• • OPT9: Hands-on Machine Learning 

Instructor: Marc Evrard

A practical oriented class, where students apply ML techniques to simple illustrative examples and then to tackle competitive challenges. It will start with an introduction to present (refresh) the ML landscape. Classes will then be articulated to successively focus on the major concepts of practical ML. Outline:

•  Introduction/refresher on ML

•  Working with real data

•  Discover and visualize the data to gain insights

•  Prepare the data for processing

•  Select and train models

• Fine-tune models

(details)[external site][ecampus]. participate to a challenge -- With PRE1 and PRE2 as prerequisite.

• • OPT13: INFORMATION THEORY

Instructor:  Guillaume Charpiat

TD: Styliani Douka

Information theory provides a theoretical ground for ML in general. 

The problems we aim at solving:

•  how to complete a sequence?  1 2 3 5 7 11 13 17 ... ?? Why is 19 more "probable"? how to justify it?

•  how to decide between two models for given data?

•  how to set a ML problem? which criterion to optimize? how to measure a solution's performance?

          and this will lead us to the following problems:

• how to quantify information?

•  is there a "natural" distribution over numbers? over models? 

• how to compress (losslessly) data? are there bounds?

In more details: 

We study the tools from Information Theory that are useful in Machine Learning. We introduce the concept of entropy, leading to a distance measure between distributions (Kullback-Leibler divergence). We then study the equivalence between compression, prediction and generation. In a third part, we get a glimpse of information geometry (Fisher metric). Last, we formalize the preference for simpler models through Kolmogorov complexity.

Chapter 1 : Entropy

Chapter 2 : Compression/Prediction/Generation equivalence

Chapter 3 : Fisher information 

Chapter 4 : Kolmogorov complexity

Course website: https://www.lri.fr/~gcharpia/informationtheory/

 (website from a previous similar class). Théorie de l'information -- With PRE1 as prerequisite

• • OPT 15: Fairness in AI

Instructor: Adrian Popescu

This course discusses theoretical and practical aspects of fairness in AI. The objectives are to: 

•  Highilight the importance of building fair AI systems and analyze the legal, ethical, social and technical challenges of this process.

•  Present the main types of biases appearing in AI systems and describe the existing approaches to mitigate them. 

•  Illustrate the practical impact of (un)fairness through impactful applications such as: personal data processing, news analysis, and face recognition.

•  Develop you critical skills with hands-on analysis of recent research paper and programming activities. 

Old content: Syllabus. With PRE1 as prerequisites. Recommended to be taken jointly with OPT 16.

• • OPT 16: Creation of a challenge in Artificial Intelligence

Instructor:  Thomas Moreau

This course aims to learn the practical tools for datascience and how to frame and solve datascience problems.

• Data wrangling

•  The scikit-learn API and Missing values

•  Metrics and unbalanced data

•  Dealing with complex data

•  Ensemble methods and hyperparameter optimization

Full syllabus: https://github.com/x-datascience-datacamp/datacamp-master

 Old content:  (Website from previous years) [ecampus].: Create a challenge (that other students will solve as a TER project). Team work in teams of 5-6 people. -- With TC0 and PRE4 (or equivalent) as prerequisite.

• • OPT 17: Hands-on Natural Language Processing

Instructors:  Kim Gerdes, Nona Naderi

•  Introduction to NLP

•  Basic concepts in NLP, tokenization, lemmatization, POS tagging,...

•  Lexical semantics, word sense disambiguation

•  Syntax and interpretations

•  Parsing

Old content: [external site]. 

• • Summer school (école thématique)

  • Internship (stage) (Coordination: Nona Naderi)

  • TER: Travail d'Etude et de Recherche (small internship). Personal work supervised by a member of the master's teaching team or in s university research lab. This work can take the form of a state of the art on a given scientific subject and / or the implementation of state algorithms for application on a given problem. This work is normally not remunerated. However, If an internship is longer than 2 months it must be remunerated. (Coordination: Nona Naderi)

• • Languages: French or English for non-native speakers.

  • I&E: Innovation and Entrepreneurship.

  • FVE: Research and Development training, mutualised with avec MIAGE d'Orsay. Formation à la Vie en Entreprise

  • Teacher responsible for I&E basics/BDlabs (M1 EIT) is: Guillaume DION

Tronc commun (TC) “classic classes", mandatory for all [AI] students, 2,5 ECTS each. 

• • TC4: PROBABILISTIC GENERATIVE MODELS (website from last year) (NEW website 2020/2021)-online-class, website- HMMs etc. -- with PRE1 and 2 as prerequisite

  • TC5: SIGNAL PROCESSING (Website from last year) -online-class, website- Traitement du signal -- with PRE2 and 4 as prerequisite

Formerly all OPTional classes, though some are now mandatory :-) 2.5 ECTS each. 

green (ML path) indicates Machine Learning of Computer Vision classes or brown (NLP path) indicate Natural Language Processing classes. 

OPT1: GRAPHICAL MODELS (details) (website from previous year) [ecampus]. Modèles graphiques pour l’accès à l'information à grande échelle -- With TC4 as prerequisite

OPT2: COMPUTER VISION (website from last year) New class, syllabus -- With TC1 and OPT4 as prerequisite.

OPT3: REINFORCEMENT LEARNING (overleaf)  [OLD external site][ecampus]. Apprentissage par renforcement -- With TC1 as prerequisite.

OPT5: AUTOMATIC SPEECH RECOGNITION AND NATURAL LANGUAGE PROCESSING [external site][ecampus].

OPT6: LEARNING THEORY AND ADVANCED MACHINE LEARNING   [external site] [ecampus]. Apprentissage avancé et théorie -- With TC1 as prerequisite

OPT7: ADVANCED OPTIMIZATION AND AUTOMATED MACHINE LEARNING  [external site][ecampus] (Formely Optimisation avancée,) -- With TC2 as prerequisite

OPT 10: IMAGE MINING -- With PRE1, 2, and 4, as prerequisites.

OPT 11: DEEP LEARNING FOR NLP  (website from previous year)   [external site][ecampus]. Natural Language Processing -- With OPT4 as prerequisite.

OPT 12: TEXT MINING AND CHATBOTS -- [external site][ecampus]- With TC3 and 6 as prerequisite.

OPT 14: MULTILINGUAL NATURAL LANGUAGE PROCESSING (details) [external site][ecampus] -- With TC4 as prerequisite.

• Communitation

• I&E: Innovation and Entrepreneurship.

• Conferences

• Teacher responsible for I&E Study (M2 EIT ) is: Alvaro PINA STRANGER

5 to 6 month internship in a research lab or a company (Coordination: Marc Evrard and Thomas Gerald).