About the seminar

Many applications in machine learning involve data represented as probability distributions. The emergence of such data requires radically novel techniques to design tractable distances and gradient flows on probability distributions over this type of (infinite-dimensional) objects. For instance, being able to flow labeled datasets is a core task for applications ranging from domain adaptation to transfer learning or dataset distillation. In this setting, we propose to represent each class by the associated conditional distribution of features, and to model the dataset as a mixture distribution supported on these classes (which are themselves probability distributions), meaning that labeled datasets can be seen as probability distributions over probability distributions. We endow this space with a metric structure from optimal transport, namely the Wasserstein over Wasserstein (WoW) distance, derive a differential structure on this space, and define WoW gradient flows. The latter enables to design dynamics over this space that decrease a given objective functional. We apply our framework to transfer learning and dataset distillation tasks, leveraging our gradient flow construction as well as novel tractable functionals that take the form of Maximum Mean Discrepancies with Sliced-Wasserstein based kernels between probability distributions. We also introduce a new sliced distance specifically designed to the space of probability over probability distributions through projections using the Busemann function.

Based on this paper and this preprint

Federated sampling algorithms have recently gained great popularity in the community of machine learning and statistics. In this talk we will present the problem of sampling in the federated learning framework and discuss variants of algorithms called Error Feedback Langevin algorithms (ELF) that are designed for this setting. In particular, we will discuss how the combinations of EF21 and EF21-P with the federated Langevin Monte-Carlo improve on prior methods.

Machine learning (ML) and artificial intelligence (AI) now automate entire systems rather than individual tasks. As such, ML/AI models are no longer responsible for a single top-line metric (e.g., prediction accuracy), but must face a growing set of potentially conflicting system requirements, such as robustness, fairness, safety, and alignment with prior knowledge. These challenges are exacerbated in uncertain, data-driven settings and further complicated by the scale and heterogeneity of modern ML/AI applications that involve from static, discriminative models (e.g., neural network classifiers) to dynamic, generative models (e.g., Langevin diffusions used for sampling). This keynote defines WHAT constitutes a requirement and explains WHY incorporating them into learning is critical. It then shows HOW to do so using constrained learning and illustrates WHEN and WHERE this approach is effective by presenting use cases in ML for science, safe reinforcement learning, and sampling. Ultimately, this talk aims to show you (WHO) that constrained learning is key to building trustworthy ML/AI systems, enabling a shift from a paradigm of artificial intelligence that is supposed to implicitly emerge from data to one of engineered intelligence that explicitly does what we want.

In this talk, I will present some results on supervised classification for continuous-time processes. The idea is to be able to propose some consistent classifier, for different type of temporal data.
The methodology relies on mimicking the Bayes rule in the multiclass classification setting, in order to derive a classifier that is consistent as the number of labeled observations increases. I will first consider the case of repeated observations of paths of stochastic differential equations driven by Brownian motion, with different labels.
In this setting, I use a plug-in technique to estimate the conditional probability that the label equals k given the observation, and I will provide some cases in which the optimal rate of convergence is achieved. I will then move on to the case of interacting particle systems of McKean–Vlasov type. This framework generalizes the previous example and raises additional challenges.
The third case I will discuss concerns multivariate Hawkes processes with
K classes, distinguished by the parameters of their intensity functions. Here, the observations are event-time data. The multivariate Hawkes process is a nice way to model interactive agents for which some events are recorded continuously. To derive a consistent classification rule in this context, I will present an Empirical Risk Minimization strategy with a refitting step using a Lasso criterion.

The main novelty of the presented results is that they do not rely on the asymptotic properties of the underlying processes, and that the considered models are high-dimensional.