Data Analytics Research Seminar

Incoming seminars

• 25 April 2024,10:30-11:45 

Vincent Fortuin (Helmholtz AI/TUM)

• • Title: Use Cases for Bayesian Deep Learning in the Age of ChatGPT

  • Abstract: Many researchers have pondered the same existential questions since the release of ChatGPT: Is scale really all you need? Will the future of machine learning rely exclusively on foundation models? Should we all drop our current research agenda and work on the next large language model instead? In this talk, I will try to make the case that the answer to all these questions should be a convinced “no” and that now, maybe more than ever, should be the time to focus on fundamental questions in machine learning again. I will provide evidence for this by presenting three modern use cases of Bayesian deep learning in the areas of self-supervised learning, interpretable additive modeling, and neural network sparsification. Together, these will show that the research field of Bayesian deep learning is very much alive and thriving and that its potential for valuable real-world impact is only just unfolding.

• 2 May 2024,10:30-11:45 

Gérard Ben Arous (NYU)

• • Title: Dynamical spectral transition for optimization in very high dimensions

  • Abstract: In recent work with Reza Gheissari (Northwestern), Aukosh Jagannath (Waterloo) we gave a general context for the existence of projected low dimensional “effective dynamics” of Stochastic Gradient Descent in very high dimensional Data Science problems. These effective dynamics (and, in particular, their so-called ‘critical regime”) define a dynamical system in finite dimensions which may be quite complex, and rules the performance of the learning algorithm. 

The next step is to understand how the system finds these “summary statistics”.  This is done in the last work with the same authors and with Jiaoyang Huang (Wharton, U-Penn). This is based on a dynamical spectral transition of Random Matrix Theory: along the trajectory of the optimization path, the Gram matrix or the Hessian matrix develop outliers which carry these effective dynamics.

I will naturally first come back to the Random Matrix Tools needed here (the behavior of the edge of the spectrum and the BBP transition).

And then illustrate the use of this point of view on a few central examples of ML:  classification for Gaussian mixtures, and the XOR task.

References:  NeurIPS 2022, Best paper award, CPAM March 2024, ICLR May 2024, and Arxiv 2310.03010.

• 23 May 2024,10:30-11:45 

Céline Duval (Université de Lille)

• • Title: Geometry of excursion sets: computing the surface area from discretized points

  • Abstract: The excursion sets of a smooth random field carries relevant information in its various geometric measures. After an introduction of these geometrical quantities showing how they are related to the parameters of the field, we focus on the problem of discretization. From a computational viewpoint, one never has access to the continuous observation of the excursion set, but rather to observations at discrete points in space. It has been reported that for specific regular lattices of points in dimensions 2 and 3, the usual estimate of the surface area of the excursions remains biased even when the lattice becomes dense in the domain of observation. We show that this limiting bias is invariant to the locations of the observation points and that it only depends on the ambiant dimension. (based on joint works with H. Biermé, R. Cotsakis, E. Di Bernardino and A. Estrade)

• 6 June 2024,10:30-11:45 

Peter Radchenko (University of Sydney)

• • Title: TBA

  • Abstract: TBA

• 13 June 2024,10:30-11:45 

Gilles Stoltz (Université Paris-Saclay)

• • Title: TBA

  • Abstract: TBA

Past seminars 2023-2024

• 5 October 2023,10:30-11:45 

Sirio Legramanti (University of Bergamo)

• • Title: Weighting covariates in Bayesian nonparametric clustering: an application to transportation networks

  • Abstract: In clustering, observed individual data are often accompanied by covariates that can assist the clustering process itself. This is the case, for example, of transportation networks, where each node has spatial coordinates, and it is often desirable that clusters of nodes are spatially cohesive. In fact, the obtained clusters may be used to inform public policy decisions, and it may be preferable that such policies are uniform over neighboring areas. Naturally, depending on the application, different notions of closeness can be used to define such neighborhoods, thus potentially requiring to transform the spatial covariates.

Motivated by real-world data about subscriptions to the public transportation system of Bergamo (Italy) and its surroundings, we propose a method to incorporate properly transformed spatial covariates into a state-of-the-art stochastic block model, while inferring the weight of covariates. (Joint work with Valentina Ghidini and Raffaele Argiento)

• 19 October 2023,10:30-11:45 

Badr-Eddine Chérief-Abdellatif (LPSM, CNRS)

• • Title: Label Shift Quantification via Distribution Feature Matching

  • Abstract: Quantification learning deals with the task of estimating the target label distribution under label shift. In this talk, we present a unifying framework, distribution feature matching (DFM), that recovers as particular instances various estimators introduced in previous literature. We derive a general performance bound for DFM procedures and extend this analysis to study robustness of DFM procedures in the misspecified setting under departure from the exact label shift hypothesis, in particular in the case of contamination of the target by an unknown distribution.

• 16 November 2023,10:30-11:45 

Nikolaus Schweizer (Tilburg University)

• • Title: Solving Maxmin Optimization Problems via Population Games

  • Abstract: The Iteratively Reweighted Least Squares (IRLS) method is well known in numerical analysis as a useful technique for solving minmax function approximation problems on a finite grid. We extend the method so that it can be applied to find maxmin solutions of more general multicriteria problems. As in the original IRLS method, the key idea is to find the maxmin decision as an optimizer of a suitably weighted sum of monotonic transformations of the criterion functions. The method is effective when transformations can be found that make the resulting weighted-sum problems easy to solve. The relevant weights are determined by an iterative scheme. For this, we use a discrete-time version of the celebrated replicator equation of evolutionary game theory, also known in machine learning as the exponential multiplicative weights algorithm. The iterative process can be viewed as the co-evolution of a population of "testers" jointly with the decision maker, which produces the maxmin solution from a symmetric Nash equilibrium in a population game. This establishes a connection to game theory that is quite different from the usual one via two-person zero-sum games. Examples are provided to show the use of the generalized IRLS method in collective investment and in decision making under uncertainty.

(Joint work with Anne Balter and Johannes M. Schumacher)

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4264811

• 14 December 2023,10:30-11:45 

Artem Prokhorov (University of Sidney)

• • Title: A machine learning attack on illegal trading

  • Abstract: We design an adaptive framework for the detection of illegal trading behavior. Its key component is an extension of a pattern recognition tool, originating from the field of signal processing and adapted to modern electronic systems of securities trading. The new method combines the flexibility of dynamic time warping with contemporary approaches from extreme value theory to explore large-scale transaction data and accurately identify illegal trading patterns. Importantly, our method does not need access to any confirmed illegal transactions for training. We use a high-frequency order book dataset provided by an international investment firm to show that the method achieves remarkable improvements over alternative approaches in the identification of suspected illegal insider trading cases.

https://doi.org/10.1016/j.jbankfin.2022.106735 

• 1 February 2024,10:30-11:45 

Gábor Lugosi (Universitat Pompeu Fabra)

• • Title: Network archaeology: a review of recent results

  • Abstract: Large networks that change dynamically over time are ubiquitous in various areas such as social networks, and epidemiology. These networks are often modeled by random dynamics which, despite being relatively simple, give a quite accurate macroscopic description of real networks. "Network archaeology"  is an area of combinatorial statistics in which one studies statistical problems of inferring the past properties of such growing networks. In this talk we discuss some simple network models and review recent results on revealing the past of the networks.

• 29 February 2024,10:30-11:45 

Claire Boyer (LPSM, Sorbonne Université)

• • Title: Some statistical insights on physics-informed machine learning

  • Abstract: Physics-informed machine learning combines the expressiveness of data-based approaches with the interpretability of physical models. In this context, we consider a general regression problem where the empirical risk is regularized by a partial differential equation that quantifies the physical inconsistency. We prove that for linear differential priors, the problem can be formulated as a kernel regression task, giving a rigorous framework to analyze physics-informed ML. In particular, the physical prior can help in boosting the estimator convergence.

The direct implementation of physics-informed kernel estimators can be tedious, and practitioners often resort to physics-informed neural networks (PINNs) instead. We offer some food for thought and statistical insight into the proper use of PINNs.

• 14 March 2024,10:30-11:45 

Matteo Barigozzi (Università di Bologna)

• • Title: Title: High-dimensional dynamic matrix factor models

  • Abstract: High-dimensional matrix-variate time series data are becoming increasingly popular in economics and finance. This has stimulated the development of matrix factor models to achieve significant dimension reduction. This paper proposes an approximate dynamic matrix factor model that accounts for the time series nature of the data, and develops an EM algorithm to perform quasi-maximum likelihood estimation of the model parameters. The algorithm is further extended to estimate the dynamic matrix factor model on a dataset with an arbitrary pattern of missing data. We prove consistency of the estimated row and column loadings matrices and of the matrix factors. The finite sample properties of the proposed estimation strategies are assessed through a large simulation study and an application to a financial dataset.

Matteo Barigozzi and Luca Trapin

• 28 March 2024,10:30-11:45 

Davide La Vecchia (University of Geneva)

• • Title: Saddlepoint techniques for the statistical analysis of time series

  • Abstract: Saddlepoint techniques provide numerically accurate, small sample approximations to the distribution of estimators and test statistics. While a complete theory on saddlepoint techniques is available in the case of independent observations, much less attention has been devoted to the time series setting. This talks contributes to fill this gap. Under short and/or long range serial dependence, for Gaussian and non Gaussian processes,  the talk shows how to derive and implement saddlepoint  approximations for Whittle's estimator, a frequency domain M-estimator. The derivation is based on the treatment of the standardized periodogram ordinates as (i.) i.d. random variables.  Comparisons of the saddlepoint techniques to other methods are presented: the numerical exercises show that the saddlepoint approximations yield accuracy improvements over extant methods, while preserving analytical tractability and avoiding resampling.  The talks starts with a gentle introduction to saddlepoint techniques in the i.i.d. setting and with a review of the basic frequency domain tools for time series analysis. The results are based on joint works with E. Ronchetti and A. Moor.