an introduction to
Optimization on smooth manifolds

I taught Riemannian optimization at EPFL in spring 2023 for the course MATH-512. The recorded lectures are available below. A semester runs for 14 weeks. Most weeks have a lecture and an exercise session, which each last 90-95 minutes (week 5 had double lectures). The students further learned the material by completing two extensive projects.

Video links bring you to an external hosting service. There, you can change the playback speed of videos (play faster, slower): see controls at the bottom-right of the video panel. You can also download the videos.

I gratefully acknowledge EPFL's CEDE (Matthew Goodman in particular) for video post-production!

Exercises and solutions prepared with Christopher Criscitiello and Timon Miehling in 2023.

Click a question to display a sketch of the answer if available. Click a sketch to display a detailed answer if available. If no sketch is available but a detailed answer is, then clicking the question displays the detailed answer.

To load exercises below, go to the Lectures tab, and click "Exercises" next to a week's header.

You may find the Manopt toolboxes helpful to use Riemannian optimization in your projects:

• Manopt in Matlab,
• PyManopt in Python,
• Manopt.jl in Julia.

For example, in Matlab, to minimize the function $f(x) = x^\top A x$ on the sphere in $\Rn$, you can write:

n = 42;
A = randsym(n);
problem.M = spherefactory(n);
problem.cost = @(x) x'*A*x;
problem.egrad = @(x) 2*A*x;  % or use autodiff: problem = manoptAD(problem);
x = trustregions(problem);   % call an overly fancy algorithm

In Python, you could use JAX (among other backends) and write:

import pymanopt
import jax

n = 42
key = jax.random.key(0)
A = jax.random.normal(key, [n, n])
A = A + A.T

manifold = pymanopt.manifolds.Sphere(n)
@pymanopt.function.jax(manifold)
def cost(x):
    return x.T @ A @ x

problem = pymanopt.Problem(manifold, cost)
optimizer = pymanopt.optimizers.TrustRegions()
result = optimizer.run(problem)

In Julia, you might write:

using Manopt, Manifolds, LinearAlgebra
n = 42
A = Symmetric(randn(n, n))
M = Sphere(n-1)
f(E, x) = x'*A*x
∇f(E, x) = 2*A*x  # define the Euclidean gradient
x = trust_regions(M, f, ∇f; objective_type=:Euclidean)

Contact: nicolas.boumal@epfl.ch.

For general questions about geometry and optimization, please use the Manopt forum, MathOverflow or Math StackExchange.

I gave a minitutorial on Riemannian optimization at SIAM Conference on Optimization 2023. The format was two lectures of 90 minutes each. The slides used during the tutorial hold a summary of the basic geometric tools and algorithms useful for optimization. You can also download the Manopt example codes for Max-Cut in Matlab. When/if videos become available, I will link to them here.

Here are an (older) one-hour video and a two-hour video covering some of those concepts. The former was a bootcamp tutorial at the Simons Institute: the slides of that video are available at that link. The two videos have mostly the same contents.

With high probability, you will enjoy the classic book Optimization Algorithms on Matrix Manifolds by Absil, Mahony and Sepulchre (Princeton University Press, 2008), also freely available online.