Descriptif des cours
Cours obligatoires
- Bayesian inference
Bayesian inference
- Data Science Lab
Data Science Lab
Ects : 4
Enseignant responsable :
Volume horaire : 24
Description du contenu de l'enseignement :
Students enrolled in this class will form groups and choose one topic among a list of proposed topics in the core areas of the master such as supervised or unsupervised learning, recommendation, game AI, distributed or parallel data-science, etc. The topics will generally consist in applying a well-established technique on a novel data-science challenge or in applying recent research results on a classical data-science challenge. Either way, each topic will come with its own novel scientific challenge to address. At the end of the module, the students will give an oral presentation to demonstrate their methodology and their findings. Strong scientific rigor as well as very good engineering and communication skills will be necessary to complete this module successfully.
Compétences à acquérir :
The goal of this module is to provide students with a hands-on experience on a novel data-science/AI challenge using state-of-the-art tools and techniques discussed during other classes of this master.
- Foundations of Machine Learning
Foundations of Machine Learning
Ects : 4
Enseignant responsable :
- FRANCIS BACH
Volume horaire : 24
Description du contenu de l'enseignement :
The course will introduce the theoretical foundations of machine learning, review the most successful algorithms with their theoretical guarantees, and discuss their application in real world problems. The covered topics are:
-
Part 1: Supervised Learning Theory: the batch setting
- Intro
- Surrogate Losses
- Uniform Convergence and PAC Learning
- Empirical Risk Minimization and ill-posed problems
- Concentration Inequalities
- Universal consistency, PAC Learnability
- VC dimension
- Rademacher complexity
- Non Uniform Learning and Model Selection
- biais-variance tradeoff
- Structural Minimization Principle and Minimum Description Length Principle
- Regularization
-
Part 2: Supervised Learning Theory and Algorithms in the Online Setting
- Foundations of Online Learning
- Beyond the Perceptron algorithm
-
Partie 3: Ensemble Methods and Kernels Methods
- SVMs, Kernels
- Kernel approximation algorithms in the primal
- Ensemble methods: bagging, boosting, gradient boosting, random forests
-
Partie 4: Algorithms for Unsupervised Learning
- Dimensionality reduction: PCA, ICA, Kernel PCA, ISOMAP, LLE
- Representation Learning
- Expectation Maximization, Latent models and Variational methods
Pré-requis recommandés :
- Linear models
Pré-requis obligatoire :
- Linear Algebra - Statistics and Probability
Compétences à acquérir :
The aim of this course is to provide the students with the fundamental concepts and tools for developing and analyzing machine learning algorithms.
Mode de contrôle des connaissances :
- Each student will have to have the role of scribe during one lecture, taking notes during the class and sending the notes to the teacher in pdf. - Final exam
Bibliographie-lectures recommandées
The most important book: - Shalev-Shwartz, S., & Ben-David, S. (2014). Understanding machine learning: From theory to algorithms. Cambridge university press. Also: - Mohri, M., Rostamizadeh, A., & Talwalkar, A. (2012). Foundations of machine learning. MIT press. - Vapnik, V. (2013). The nature of statistical learning theory. Springer science & business media. - Bishop Ch. (2006). Pattern recognition and machine learning. Springer - Friedman, J., Hastie, T., & Tibshirani, R. (2001). The elements of statistical learning (Vol. 1, No. 10). New York, NY, USA:: Springer series in statistics. - James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112). New York: springer.
- Large dimensional statistics
Large dimensional statistics
- Optimal transport (ENS)
Optimal transport (ENS)
Ects : 4
Volume horaire : 24
Description du contenu de l'enseignement :
Optimal transport (OT) is a fundamental mathematical theory at the interface between optimization, partial differential equations and probability. It has recently emerged as an important tool to tackle a surprisingly large range of problems in data sciences, such as shape registration in medical imaging, structured prediction problems in supervised learning and training deep generative networks. This course will interleave the description of the mathematical theory with the recent developments of scalable numerical solvers. This will highlight the importance of recent advances in regularized approaches for OT which allow one to tackle high dimensional learning problems.
The course will feature numerical sessions using Python.
- Motivations, basics of probabilistic modeling and matching problems.
- Monge problem, 1D case, Gaussian distributions.
- Kantorovitch formulation, linear programming, metric properties.
- Shrödinger problem, Sinkhorn algorithm.
- Duality and c-transforms, Brenier’s theory, W1, generative modeling.
- Semi-discrete OT, quantization, Sinkhorn dual and divergences
- Optimization for Machine Learning
Optimization for Machine Learning
Ects : 4
Enseignant responsable :
Volume horaire : 24
Description du contenu de l'enseignement :
Optimization has long been a fundamental component for modeling and solving classical machine learning problems such as linear regression and SVM classification. It also plays a key role in the training of neural networks, thanks to the development of efficient numerical tools tailored to deep learning. This course is concerned with developing optimization algorithms for learning tasks, and will consist of both lectures and hands-on sessions in Python. The course will begin by an introduction to the various problem formulations arising in machine and deep learning, together with a refresher on key mathematical concepts (linear algebra, convexity, smoothness). The course will then describe the main algorithms for optimization in data science (gradient descent, stochastic gradient) and their theoretical properties. Finally, the course will focus on the challenges posed by implementing these methods in a deep learning and large-scale environment (automatic differentiation, distributed calculations, regularization).
Compétences à acquérir :
- Understand the nature and structure of optimization problems arising in machine learning.
- Select an algorithm tailored to solving a particular instance among those seen in class based on theoretical and practical concerns.
- Experience the practical challenges in implementing an optimization scheme in a learning setting.
Bibliographie-lectures recommandées
- L. Bottou, F. E. Curtis and J. Nocedal. Optimization Methods for Large-Scale Machine Learning. SIAM Review, 2018.
- S. J. Wright and B. Recht. Optimization for Data Analysis. Cambridge University Press, 2022.
- Reinforcement learning
Reinforcement learning
Ects : 4
Enseignant responsable :
- OLIVIER CAPPE
Volume horaire : 24
Description du contenu de l'enseignement :
- Models: Markov decision processes (MDP), multiarmed bandits and other models
- Planning: finite and infinite horizon problems, the value function, Bellman equations, dynamic programming, value and policy iteration
- Basic learning tools: Monte Carlo methods, temporal-difference learning, policy gradient
- Probabilistic and statistical tools for RL: Bayesian approach, relative entropy and hypothesis testing, concentration inequalities
- Optimal exploration in multiarmed bandits: the explore vs exploit tradeoff, lower bounds, the UCB algorithm, Thompson sampling
- Extensions: Contextual bandits, optimal exploration for MDP
Compétences à acquérir :
Reinforcement Learning (RL) refers to scenarios where the learning algorithm operates in closed-loop, simultaneously using past data to adjust its decisions and taking actions that will influence future observations. Algorithms based on RL concepts are now commonly used in programmatic marketing on the web, robotics or in computer game playing. All models for RL share a common concern that in order to attain one's long-term optimality goals, it is necessary to reach a proper balance between exploration (discovery of yet uncertain behaviors) and exploitation (focusing on the actions that have produced the most relevant results so far).
The methods used in RL draw ideas from control, statistics and machine learning. This introductory course will provide the main methodological building blocks of RL, focussing on probabilistic methods in the case where both the set of possible actions and the state space of the system are finite. Some basic notions in probability theory are required to follow the course. The course will imply some work on simple implementations of the algorithms, assuming familiarity with Python.
Mode de contrôle des connaissances :
- Individual homework (in Python)
- Final exam
Bibliographie-lectures recommandées
Bibliographie, lectures recommandées
- M. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, 1994.
- R. Sutton and A. Barto. Introduction to Reinforcement Learning. MIT Press, 1998.
- C. Szepesvari. Algorithms for Reinforcement Learning. Morgan & Claypool Publishers, 2010.
- T. Lattimore and C. Szepesvari. Bandit Algorithms. Cambridge University Press. 2019.
Cours optionnels - 5 cours à choisir parmi :
- Bayesian machine learning
Bayesian machine learning
Ects : 4
Enseignant responsable :
Volume horaire : 24
Description du contenu de l'enseignement :
Bayesian Nonparametrics:
- Introduction
- The Dirichlet Process
- Infinite Mixture models
- Posterior Sampling
- Models beyond the Dirichlet Process
- Gaussian Processes
- Selected applications
Bayesian Deep Learning
- Why do we want parameter uncertainty
- Priors for Bayesian neural networks
- Posterior inference
- Martingale Posteriors and generalised Bayesian Inference
Pré-requis obligatoire :
- Bayesian statistics
- Markov Chain Monte Carlo
Compétences à acquérir :
Essentials of Bayesian Nonparametrics, main concepts for Bayesian Deep Learning
Mode de contrôle des connaissances :
Final exam and homework
Bibliographie-lectures recommandées
- Hjort NL, Holmes C, Müller P, Walker SG, editors. Bayesian nonparametrics. Cambridge University Press; 2010 Apr 12.
- Ghosal S, Van der Vaart AW. Fundamentals of nonparametric Bayesian inference. Cambridge University Press; 2017 Jun 26.
- Williams CK, Rasmussen CE. Gaussian processes for machine learning. Cambridge, MA: MIT press; 2006.
- Many references at www.gatsby.ucl.ac.uk/~porbanz/npb-tutorial.html
- Murphy KP. Probabilistic machine learning: Advanced topics. MIT press; 2023 Aug 15.
- Fong E, Holmes C, Walker SG. Martingale posterior distributions. Journal of the Royal Statistical Society Series B: Statistical Methodology. 2023 Nov;85(5):1357-91.
- Computational statistics methods and MCMC
Computational statistics methods and MCMC
- Data acquisition, extraction and storage
Data acquisition, extraction and storage
Ects : 4
Enseignant responsable :
Volume horaire : 24
Description du contenu de l'enseignement :
The objective of this course is to present the principles and techniques used to acquire, extract, integrate, clean, preprocess, store, and query datasets, that may then be used as input data to train various artificial intelligence models. The course will consist on a mix of lectures and practical sessions. We will cover the following aspects:
- Web data acquisition (Web crawling, Web APIs, open data, legal issues)
- Information extraction from semi-structured data
- Data cleaning and data deduplication
- Data formats and data models
- Storing and processing data in databases, in main memory, or in plain files
- Introduction to large-scale data processing with MapReduce and Spark
- Ontology-based data access
Pré-requis obligatoire :
Basics of computer science and computer engineering (algorithms, databases, programming, logics, complexity).
Compétences à acquérir :
Understanding:
- how to acquire data from a variety of sources and in a variety of formats
- how to extract structured data from unstructured or semi-structured data
- how to format, integrate, clean data sets
- how to store and access data sets
Mode de contrôle des connaissances :
Project (50% of the grade) and in-class written assessment (50% of the grade)
En savoir plus sur le cours :
- Deep learning for image analysis
Deep learning for image analysis
Ects : 4
Enseignant responsable :
- Etienne DECENCIERE
Volume horaire : 24
Description du contenu de l'enseignement :
Deep learning has achieved formidable results in the image analysis field in recent years, in many cases exceeding human performance. This success opens paths for new applications, entrepreneurship and research, while making the field very competitive.
This course aims at providing the students with the theoretical and practical basis for understanding and using deep learning for image analysis applications.
Program to be followed The course will be composed of lectures and practical sessions. Moreover, experts from industry will present practical applications of deep learning. Lectures will include:
- Artificial neural networks, back-propagation algorithm - Convolutional neural networks - Design and optimization of a neural architecture - Analysis of neural network function - Image classification and segmentation - Auto-encoders and generative networks - Transformers - Current research trends and perspectives
During the practical sessions, the students will code in Python, using Keras or Pytorch. They will be confronted with the practical problems linked to deep learning: architecture design; optimization schemes and hyper-parameter selection; analysis of results.
Pré-requis obligatoire :
- Linear algebra, basic probability and statistics
- Python
Compétences à acquérir :
Deep learning for image analysis: theoretical foundations and applications
Mode de contrôle des connaissances :
Practical session and exam
- Dimension reduction and manifold learning
Dimension reduction and manifold learning
Ects : 4
Volume horaire : 24
Description du contenu de l'enseignement :
Modern machine learning typically deals with high-dimensional data. The fields concerned are very varied and include genomics, image, text, time series, or even socioeconomic data where more and more unstructured features are routinely collected. As a counterpart of this tendency towards exhaustiveness, understanding these data raises challenges in terms of computational resources and human understandability. Manifold Learning refers to a family of methods aiming at reducing the dimension of data while preserving certain of its geometric and structural characteristics. It is widely used in machine learning and experimental science to compress, visualize and interpret high-dimensional data. This course will provide a global overview of the methodology of the field, while focusing on the mathematical aspects underlying the techniques used in practice.
Pré-requis obligatoire :
Linear algebra, basic probability theory, statistics, Python coding
Compétences à acquérir :
- Curse of dimensionality, manifold hypothesis and intrinsic dimension(s) - Multidimensional scaling - Linear dimension reduction (random projections, principal component analysis) - Non-linear spectral methods (kernel PCA, ISOMAP, MVU, Laplacian eigenmaps) - Ad-hoc distance-preserving methods (diffusion maps, LLE) - Probabilistic dimension reduction and clustering (SNE, UMAP) - Neural network-based dimensionality reduction
Bibliographie-lectures recommandées
- Ghojogh, B., M. Crowley, F. Karray, and A. Ghodsi (2023). Elements of dimensionality reduction and manifold learning - Lee, J. A., M. Verleysen, et al. (2007). Nonlinear dimensionality reduction
- Graphical models
Graphical models
Ects : 4
Enseignant responsable :
Volume horaire : 24
Description du contenu de l'enseignement :
This course provides an introduction to causal inference. It covers both the Neyman–Rubin potential outcomes framework and Pearl’s do-calculus. The former is used to introduce the fundamental problem of causal inference and the notion of counterfactuals. The core hypotheses needed for causal identification of average treatment effects are presented: (conditional) exchangeability, positivity, and consistency. Estimation based on generalised linear models and on machine learning approaches is explored, including the double-machine learning approach.
The second part of the course covers Pearl’s do-calculus. The course introduces graphical models, with a focus on directed models, followed by structural causal models. The simple Markovian case is used to link this framework to the potential outcomes one and to derive classical techniques such as the back-door criterion. The semi-Markovian case is then explored as the general way of representing causal hypotheses in the presence of unobserved confounding variables. Identification is revisited in the light of the do-calculus and of the IDC algorithm.
The final part of the course reviews causal discovery algorithms and open research questions.
Compétences à acquérir :
This course is an introduction to causal inference with a strong emphasis on the use of graphical models. After the course, the students should be able
- to apply consistent average treatment effect estimation procedures
- to turn causal hypotheses into structural causal models
- to analyse graphical models to determine independence structures
- to use do-calculus and the IDC algorithm to identify causal estimands
- Kernel methods
Kernel methods
Ects : 4
Volume horaire : 24
Description du contenu de l'enseignement :
Reproducing kernel Hilbert spaces et le “ kernel trick ” Théorème de représentation Kernel PCA Kernel ridge regression Support vector machines Noyaux sur les semigroupes Noyaux pour le texte, les graphes, etc.
Compétences à acquérir :
Présenter les bases théoriques et des applications des méthodes à noyaux en apprentissage.
- Large language models
Large language models
Ects : 4
Enseignant responsable :
Volume horaire : 24
Description du contenu de l'enseignement :
The course focuses on modern and statistical approaches to NLP.
Natural language processing (NLP) is today present in some many applications because people communicate most everything in language : post on social media, web search, advertisement, emails and SMS, customer service exchange, language translation, etc. While NLP heavily relies on machine learning approaches and the use of large corpora, the peculiarities and diversity of language data imply dedicated models to efficiently process linguistic information and the underlying computational properties of natural languages.
Moreover, NLP is a fast evolving domain, in which cutting-edge research can nowadays be introduced in large scale applications in a couple of years.
The course focuses on modern and statistical approaches to NLP: using large corpora, statistical models for acquisition, disambiguation, parsing, understanding and translation. An important part will be dedicated to deep-learning models for NLP.
- Introduction to NLP, the main tasks, issues and peculiarities - Sequence tagging: models and applications - Computational Semantics - Syntax and Parsing - Deep Learning for NLP: introduction and basics - Deep Learning for NLP: advanced architectures - Deep Learning for NLP: Machine translation, a case study
Pré-requis recommandés :
pytorch
Compétences à acquérir :
- Skills in Natural Language Processing using deep-learning
- Understand new architectures
Bibliographie-lectures recommandées
References - Costa-jussà, M. R., Allauzen, A., Barrault, L., Cho, K., & Schwenk, H. (2017). Introduction to the special issue on deep learning approaches for machine translation. Computer Speech & Language, 46, 367-373. - Dan Jurafsky and James H. Martin. Speech and Language Processing (3rd ed. draft): web.stanford.edu/~jurafsky/slp3/ - Yoav Goldberg. A Primer on Neural Network Models for Natural Language Processing: u.cs.biu.ac.il/~yogo/nnlp.pdf - Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning: www.deeplearningbook.org
- Mixing times of Markov chains
Mixing times of Markov chains
Ects : 4
Enseignant responsable :
Volume horaire : 24
Description du contenu de l'enseignement :
How many times must one shuffle a deck of 52 cards? This course is a self-contained introduction to the modern theory of mixing times of Markov chains. It consists of a guided tour through the various methods for estimating mixing times, including couplings, spectral analysis, discrete geometry, and functional inequalities. Each of those tools is illustrated on a variety of examples from different contexts: interacting particle systems, card shuffling, random walks on groups, graphs and networks, etc. Finally, a particular attention is devoted to the celebrated cutoff phenomenon, a remarkable but still mysterious phase transition in the convergence to equilibrium of certain Markov chains.
Compétences à acquérir :
See the webpage of the course.
Mode de contrôle des connaissances :
Final written exam, in class.
En savoir plus sur le cours :
www.ceremade.dauphine.fr/~salez/mix.html
Bibliographie-lectures recommandées
See the webpage of the course.
- Non-convex inverse problems
Non-convex inverse problems
Ects : 4
Enseignant responsable :
Volume horaire : 24
Description du contenu de l'enseignement :
An inverse problem is a problem where the goal is to recover an unknown object (typically a vector with real coordinates, or a matrix), given a few ``measurements'' of this object, and possibly some information on its structure. In this course, we will discuss examples of such problems, motivated by applications as diverse as medical imaging, optics and machine learning. We will especially focus on the questions: which algorithms can we use to numerically solve these problems? When and how can we prove that the solutions returned by the algorithms are correct? These questions are relatively well understood for convex inverse problems, but the course will be on non-convex inverse problems, whose study is much more recent, and a very active research topic.
The course will be at the interface between real analysis, statistics and optimization. It will include theoretical and programming exercises.
Compétences à acquérir :
Understand what is a non-convex inverse problems; get some familiarity with the most classical algorithms to solve them, and with algorithms for general non-convex optimization
PSL Week
Bloc stage
- Stage
Stage
Ects : 10
Modalités pédagogiques
Les cours sont dispensés au 16 bis rue de l'Estrapade, 75005 Paris.
Le Master IASD commence par un semestre de tronc commun consacré aux disciplines fondamentales de l’IA et des sciences des données, qui comprend quatre cours communs et trois cours spécifiques à chaque parcours. À la fin du premier semestre, les étudiants choisissent six cours d’approfondissement pour le second semestre, dont une semaine intensive PSL permettant l’ouverture thématique vers d’autres disciplines ou applications. L’année se poursuit par un stage effectué dans un laboratoire de recherche académique ou industriel et se conclut en septembre par la rédaction d’un mémoire et sa soutenance publique.
Le Master IASD se compose d’un semestre de tronc commun sur les disciplines fondamentales de l’IA (de septembre à décembre; 7 cours obligatoires, soit 168h de formation – 28 ETCS) suivi d’un semestre d’options (de janvier à mars; 6 cours optionnel, 140h de formation – 22 ECTS) et d’un stage de recherche (d’avril à septembre; 10 ETCS) effectué dans un laboratoire académique ou une entreprise. Le tronc commun comporte sept cours obligatoires tandis que le second semestre permet d’approfondir six matières à choisir parmi une vingtaine d’options, dont une semaine intensive PSL rattachée au programme transverse DATA. Des cours optionnels de remise à niveau sur les fondements probabilistes et la programmation sont proposés avant le début des cours de tronc commun au début du mois de septembre.
Les parcours IASD - Mathématiques et IASD - Informatique partagent quatre cours communs au premier semestre et se distinguent par trois cours qui sont spécifiques à chaque parcours. Les cours spécifiques d’un parcours peuvent éventuellement être choisis comme option dans l’autre parcours, dans la limite de deux options, au maximum, suivies au premier semestre.
Stages et projets tutorés
Un stage de 5 mois est obligatoire, à partir du début du mois d’avril.
Pour les étudiantes et les étudiants :comment trouver un stage, et obtenir la convention ?
Pour trouver un stage, vous pouvez consulter la liste des propositions de stage, ou démarcher vous mêmes les laboratoires ou entreprises qui vous intéressent. Dans le second cas, vous devez charger votre sujet de stage sur la plateforme pédagogique du master à cette adresse en spécifiant votre nom dans le champ commentaire pour que nous sachions que le sujet est pour vous. (Merci de ne pas envoyer votre sujet par mail.)
Une fois que vous avez chargé votre sujet de stage, vous pouvez pouvez remplir le formulaire dans l’application ESUP Stage sur le site de Dauphine – PSL pour obtenir votre convention de stage.
Pour plus d'info sur la présentation des stages.
Pour les encadrants :comment proposer un stage aux étudiantes et étudiants du master IASD ?
Si vous faites partie d’un laboratoire de recherche, ou d’un département R&D, vous pouvez proposer un sujet de stage en cliquant ici. Le stage doit être en relation avec l’un des sujets abordés dans le programme du master.
Gratification : En France, les stages de plus de 2 mois doivent s’accompagner d’une gratification. Consultez le simulateur pour calculer la rémunération des stagiaires.
Des programmes nourris par la recherche
Les formations sont construites au contact des programmes de recherche de niveau international de Dauphine, qui leur assure exigence et innovation.
La recherche est organisée autour de 6 disciplines toutes centrées sur les sciences des organisations et de la décision.
En savoir plus sur la recherche à Dauphine
A lire sur Dauphine Éclairages
notre média scientifique
Ancrer l’intelligence artificielle dans le travail réel : explicabilité située et responsabilité managériale
Synthèse de l’intervention de Yann Ferguson lors de la soirée de lancement du Cercle Travail en transitions, le 5 février...
50 nuances de vert : les négociations de la transition sociale et écologique en entreprise
Synthèse de l’intervention de Camille Dupuy lors de la soirée de lancement du Cercle Travail en transitions, le 5 février...
Pourquoi et comment démocratiser l’entreprise ?
Tout au long du XIXe siècle, l’idée que les travailleurs devraient être associés à la production et à la propriété des...