Continuous Optimization
Enseignant responsable :
Volume horaire : 24Description du contenu de l'enseignement :
This course will review the mathematical foundations of convex/continuous (iterative) optimization methods. We will focus on the theory and mathematical analysis of a few algorithmic methods and showcases some modern applications of a broad range of optimization techniques. The course will be composed of classical lectures and one numerical session in Python. The first part covers the basic methods of smooth optimization (gradient descent) and convex optimization (optimality condition, constrained optimization, duality) with some general approach (monotone operators) and a focus on convergence rates. We will then address more advanced methods (non-smooth optimization and proximal methods, stochastic gradient descent).
Compétence à acquérir :
The objective of this course is to introduce the students to classical and modern methods for the optimization of (mostly convex) objectives, possibly nonsmooth or high dimensional. These arise in areas such as learning, finance or signal processing.
Mode de contrôle des connaissances :
Examen écrit
Bibliographie, lectures recommandées
Exemples de livres généraux sur l'optimisation (souvent convexe) couvrant des aspects à la fois théoriques (complexité) et pratique (implémentations):
Boris Polyak: Introduction to optimization, (1987).
J.-B. Hiriart-Urruty and C. Lemarechal, Convex Analysis and Minimization Algorithms (1993).
Yurii Nesterov: Introductory lectures on convex optimization, 2004 / Lectures on convex optimization 2018
Jorge Nocedal and Stephen J. Wright: Numerical Optimization, 2006.
Dimitri Bertsekas: Convex Optimization Algorithms. Athena Scientific 2015.
Amir Beck: First-Order Methods In Optimization, 2019.
R. Tyrell Rockafellar: Convex analysis, 1970 (1997).
H. Bauschke and P.L. Combettes: Convex analysis and monotone operator theory in Hilbert spaces (Springer 2011)
Ivar Ekeland and Roger Temam: Convex analysis and variational problems, 1999.
Juan Peypouquet: Convex Optimization in Normed Spaces, 2015