This seminar aims to increase the links between the different laboratories in Saclay in the field of Applied Maths, Statistics and Machine Learning. The Seminar is organized every first Tuesday of the month with 2 presentations followed by a small refreshment. The localization of the seminar will change to accommodate the different labs.
Organization
Due to access restriction, you need to register for the seminar. A link is provided in the description and should also be sent with the seminar announcement. It will also help us organize for the food quantities. If you think you will come, please register! (even if you are unsure)
To not miss the next seminar, please subscribe to the announcement mailing list palaisien@inria.fr.
You can also add the calendar from the seminar to your own calendar (see below).
Next seminars
REGISTER
07 May 2025, 12h At Inria Saclay - Amphi Sophie Germain
Tony Silveti-Falls-Training Deep Learning Models with Norm-Constrained LMOs
In this talk, I discuss optimization methods that leverage the linear minimization oracle (LMO) over a norm-ball and their application to training huge neural networks. We propose a new stochastic family of algorithms that uses the LMO to adapt to the geometry of the problem and, perhaps surprisingly, show that they can be applied to unconstrained problems. The resulting update rule unifies several existing optimization methods under a single framework. Furthermore, we propose an ...
In this talk, I discuss optimization methods that leverage the linear minimization oracle (LMO) over a norm-ball and their application to training huge neural networks. We propose a new stochastic family of algorithms that uses the LMO to adapt to the geometry of the problem and, perhaps surprisingly, show that they can be applied to unconstrained problems. The resulting update rule unifies several existing optimization methods under a single framework. Furthermore, we propose an explicit choice of norm for deep architectures, which, as a side benefit, leads to the transferability of hyperparameters across model sizes. Experimentally, we demonstrate significant speedups on nanoGPT training without any reliance on Adam. The proposed method is memory-efficient, requiring only one set of model weights and one set of gradients, which can be stored in half-precision.
Erwan Allys-Generative models and component separations for physical fields with Scattering Transforms
Scattering transform statistics have led to recent advances in the modelling of physical processes. These statistics, which are inspired by neural networks but can be estimated without a training step, allow quantitative modelling of physical processes, in a maximum entropy framework, even from very small data sets. After introducing these models and quantitatively demonstrating their quality on several examples, I will discuss how they can form the basis of new algorithms for inverse problems a
Scattering transform statistics have led to recent advances in the modelling of physical processes. These statistics, which are inspired by neural networks but can be estimated without a training step, allow quantitative modelling of physical processes, in a maximum entropy framework, even from very small data sets. After introducing these models and quantitatively demonstrating their quality on several examples, I will discuss how they can form the basis of new algorithms for inverse problems and component separation. In particular, I will show how they can be used to separate components even in a very limited data regime and without physically-driven priors of the component of interest, with examples of applications to astrophysical data.
Scientific Committee
The program and the organization of this seminar is driven by a scientific committee composed of members of the different laboratories in Saclay. The members of the committee are currently:
