Numerical optimization
Enseignant responsable :
Volume horaire : 48Description du contenu de l'enseignement :
Numerical Optimisation
1. Introduction : a review of basic concepts in optimisation
(a) Optimality conditions, algorithms, convergence rates.
2. First part : Unconstrained optimisation-deterministic methods
(a) A crash course on gradient descent for smooth functions.
(b) The link with gradient flows.
(c) The case of non-convex functions.
(d) Acceleration of gradient descents.
(e) Newton and quasi-Newton methods.
(f) Complement : Back-propagation and machine learning.
3. Second part : Constrained optimisation-deterministic methods
(a) Penalisation method.
(b) The projected gradient method.
(c) Lagrange multipliers and duality-the interior point method.
4. Third part : Unconstrained optimisation-an introduction to stochastic methods
(a) Basic concepts in stochastic gradient descent. Convergence of the algorithm.
(b) Acceleration of stochastic gradient descent.
(c) (Mini)Batches.
Compétence à acquérir :
Mastering traditional techniques in numerical optimisation.