Program
Workshop 1: High-Dimensional Probability and Statistics in ML
August 22
10:00-11:30
Short introduction to Theme Semester, talk by Babak Hassibi (TBA)
11:30-13:00
Lunch (not included in the program)
13:00-14:00
Johan Jonasson, Chalmers
Noise sensitivity/stability for deep Boolean neural nets.
Abstract: A well-known and ubiquitous property of neural net classifiers is that they can be fooled into misclassifying some objects by changing the input in tiny ways that are indistinguishable for the human eye. These changes can be adversarial, but sometimes they can be just random noise. This makes it interesting to ask if this property is something that almost all neural nets have and, when they do, why that is. There are good heuristic explanations, but to prove mathematically rigorous results seems very difficult in general. Here we prove some first results on various toy models. We treat our questions within the framework of the established field of noise sensitivity/stability. What we prove can roughly be stated as:
A sufficiently deep fully connected network with sufficiently wide layers and iid Gaussian weights is noise sensitive, i.e. an arbitrarily small random noise makes the predicted class of a binary input string before and after the noise is added virtually independent. If one imposes correlations on the weights corresponding to the same input features, this still holds unless the correlation is very close to 1.
Neural nets consisting of only convolutional layers may or may not be noise sensitive and we present examples of both behaviours.
14:00-15:00
Devdatt Dubhashi, Chalmers
Can DL theory help to make neural networks leaner and faster?
Abstract: Despite their great success, a major issue with neural networks that is an obstacle to more widespread use is that the neural networks with state of the art performance are huge (with millions and even billions of parameters) and hence very demanding of computing and energy resources. A very active area of research is to prune these networks to a fraction of their original size without suffering a big hit in performance. Recent pruning methods have been guided by DL theory such as the neural tangent kernel. We will discuss some of these methods and report on our experience with a startup company from Chalmers.
15:00-16:00
Coffee break
16:00-17:00
Talk by Samet Oymak, UC Riverside
Understanding Overparameterization through Feature Covariance and High-dimensional Analysis
Abstract: An overarching goal in machine learning is to enable accurate statistical inference in the setting where the sample size is less than the number of parameters. This overparameterized setting is particularly common in deep learning where it is typical to train large neural nets with relatively smaller sample sizes and little concern of overfitting. In this talk, we highlight how structure within data is a catalyst for the empirical success of these large models. After linking deep nets to linear models, we show that the eigen-structure of the feature covariance can help explain empirical phenomena such as noise robustness, double descent curve, model compression, and the benefits of perfectly-fitting to the training data. In particular, we highlight that a typical feature covariance has a spiked structure with few large eigenvalues and many smaller ones. We proceed to discuss: (1) For data with label noise: Regularization is useful to restrict the optimization process to large eigen-directions and reduce overfitting and (2) For (mostly) noiseless data: Incorporating small eigen-directions is crucial for striking a good bias/variance tradeoff. This in turn explains why larger models work better despite perfect-fitting with no regularization. Finally, we explain how our high-dimensional analysis framework based on gaussian process theory facilitates these findings.
Bio: Samet Oymak is an assistant professor of Electrical and Computer Engineering at the University of California, Riverside. Prior to UCR, he spent three years at Google and in algorithmic finance. During his postdoc, he was at UC Berkeley as a Simons Fellow and a member of AMPLab. He obtained his bachelor's degree from Bilkent University in 2009 and PhD degree from Caltech in 2015. At Caltech, he received the Charles Wilts Prize for the best departmental thesis. At UCR, he received an NSF CAREER award as well as a Research Scholar award from Google. Website: intra.ece.ucr.edu/~oymak
August 23
9:30-10:30
Alfred Hero, University of Michigan
Screening for correlations in ultra-high dimension
Abstract: We develop a unifying approach to screening for correlations among p covariates from a finite number n of empirical observations in the ultra-high dimensional regime where n is fixed and p is large. In such ultra-high dimensional regimes thresholding methods for screening are more practical than optimization methods such as maximum likelihood. Our thresholding approach attains universal limit laws and phase transitions under a relaxation of the standard block-sparse assumptions on the covariance matrix. By making connections to random geometric graphs, the total number of highly correlated or partial correlated variables are shown to have a novel compound Poisson limit as p approaches infinity. The unifying framework also demonstrates an important duality between correlation and partial correlation screening with important theoretical and practical consequences.
10:30-11:30
Yue M. Lu, Harvard University
Exploring and Exploiting the Universality Phenomena in High-Dimensional Estimation and Learning
Abstract: Universality is a fascinating high-dimensional phenomenon. It points to the existence of universal laws that govern the macroscopic behavior of wide classes of large and complex systems, despite their differences in microscopic details. The notion of universality originated in statistical mechanics, especially in the study of phase transitions. Similar phenomena have been observed in probability theory, dynamical systems, random matrix theory, and number theory.
In this talk, I will present some recent progresses in rigorously understanding and exploiting the universality phenomena in the context of statistical estimation and learning on high-dimensional data. Examples include spectral methods for high-dimensional projection pursuit, statistical learning based on kernel and random feature models, approximate message passing algorithms, and regularized linear regression on highly structured, strongly correlated, and even (nearly) deterministic data matrices. Together, they demonstrate the robustness and wide applicability of the universality phenomena.
Bio: Yue M. Lu attended the University of Illinois at Urbana-Champaign, where he received the M.Sc. degree in mathematics and the Ph.D. degree in electrical engineering, both in 2007. He is currently Gordon McKay Professor of Electrical Engineering and of Applied Mathematics at Harvard University. He is also fortunate to have held visiting appointments at Duke University in 2016 and at the École Normale Supérieure (ENS) in 2019. His research interests include the mathematical foundations of statistical signal processing and machine learning in high dimensions.
11:30- 13:00
Lunch (not included in the program)
13:00-13:30
Fredrik Johansson, Chalmers
Efficient learning of prediction models using time-series privileged information
Abstract: In domains where sample sizes are limited, efficient learning is critical. Yet, there are machine learning problems where standard practice routinely leaves substantial information unused. One example is prediction of an outcome at the end of a time series based on variables collected at a baseline time point, for example, the 30-day risk of mortality for a patient upon admission to a hospital. In applications, it is common that intermediate samples, collected between baseline and end points, are discarded, as they are not available as input for prediction when the learned model is used. We say that this information is privileged, as it is available only at training time. In this talk, we show that making use of privileged information from intermediate time series can lead to much more efficient learning. We give conditions under which it is provably preferable to classical learning, and a suite of empirical results to support these findings.
13:30-14:00
Henrik Imberg , Chalmers
Unbiased Active Learning
Abstract: Data subsampling has become increasingly used in the statistics and machine learning community to overcome practical, economical and computational bottlenecks in modern large scale inference problems. Within this setting, active learning is a branch of machine learning dedicated to task of optimal selection of training data points from a large collection of unlabelled data. In this talk, we give a brief introduction to active learning and demonstrate its potential for common machine learning tasks. We also highlight the drawbacks of active learning, such as the inherent risk of bias, and show how this can be avoided through unbiased active learning
14:00-15:00
Pragya Sur, Harvard University
A modern central limit theorem for the classical augmented IPW estimator: variance inflation, cross-fit covariance and beyond
Abstract: Estimating the average treatment effect (ATE) is a central problem in causal inference. Modern advances in the field studied estimation and inference for the ATE in high dimensions through a variety of approaches. Doubly robust estimators such as the augmented inverse probability weighting (AIPW) form a popular approach in this context. However, the high-dimensional literature surrounding these estimators relies on sparsity conditions, either on the outcome regression (OR) or the propensity score (PS) model. This talk will introduce a new central limit theorem for the classical AIPW estimator, that applies agnostic to such sparsity-type assumptions. Specifically, we will study properties of the cross-fit version of the estimator under well-specified OR and PS models, and the common modern regime where the number of features and samples are both large and comparable. In this regime, under assumptions on the covariate distribution, our CLT will uncover two crucial phenomena among others: (i) the cross-fit AIPW exhibits a substantial variance inflation that can be precisely quantified in terms of the signal-to-noise ratio and other problem parameters, (ii) the asymptotic covariance between the estimators used while cross-fitting is non-negligible even on the root-n scale. These findings are strikingly different from their classical counterparts, and open a vista of possibilities for studying similar other high-dimensional effects. On the technical front, our work utilizes a novel interplay between three distinct tools—approximate message passing theory, the theory of deterministic equivalents, and the leave-one-out approach. Time permitting, I will outline some of these techniques. This is based on joint work with Kuanhao Jiang, Rajarshi Mukherjee, and Subhabrata Sen.
Bio: Pragya Sur is an Assistant Professor in the Statistics Department at Harvard University. Her research broadly spans high-dimensional statistics and statistical machine learning. A major part of her work focuses on developing the theoretical underpinnings of statistical inference procedures applicable for high-dimensional data. She simultaneously works on the statistical properties of modern machine learning algorithms, in particular, ensemble learning algorithms. Recently, she has been interested in developing theory and methods for causal inference in high dimensions. On the applied side, she finds interest in developing computationally scalable statistical methods with a focus on problems arising from statistical genetics. Her current research is supported by an NSF DMS Award and a William F. Milton Fund Award. Previously, she spent a year as a postdoctoral fellow at the Center for Research on Computation and Society at Harvard. She completed a Ph.D. in Statistics in 2019 from Stanford University, where she received the Ric Weiland Graduate Fellowship (2017-2019) and the 2019 Theodore W. Anderson Theory of Statistics Dissertation Award.
15:00-16:00
Coffee Break
16:00-17:00
David Bosch, Chalmers
Analyzing Random Feature Regression: Precise Asymptotic Expressions for Generic Regularization
Abstract: Gaussian comparison theorems and the principle of universality have provided a great means for exploring statistical machine learning techniques in high dimensions. In this talk, we present a novel application of this strategy to the problem of regularized random feature regression, which has gained popularity in the recent years. For this reason, we employ the well-known Convex Gaussian Min-Max Theorem (CGMT) and overcome its notorious difficulty with correlated features, by a novel scheme where the CGMT is utilized multiple times. Using this approach we present analytic expressions for the case of random feature regression under l1 regularization, also known as LASSO.
August 24
9:30-11:30
Q/A panel Discussion
Workshop 2: Theory of Deep Learning
August 29
09:30-10:30
Keynote talk by Babak Hassibi
Stochastic Mirror Descent, Implicit and Explicit Regularization
Abstract: Stochastic descent methods are the workhorse of deep learning. In fact, it is now widely recognized that the success of deep learning is not only due to the special deep architecture of the models, but also due to the behavior of the stochastic descent methods used, which play a key role in reaching "good" solutions that generalize well to unseen data. In an attempt to shed some light on why this is the case, we revisit some minimax properties of stochastic gradient descent (SGD)---originally developed for linear models in the context of H-infinity control in the 1990's---and extend them to general stochastic mirror descent (SMD) algorithms for general nonlinear models. These minimax properties can be used to explain the convergence and implicit-regularization behavior of the algorithms when the linear regression problem is over-parametrized (In what is now being called the "interpolating regime"). In the nonlinear setting, exemplified by training a deep neural network, we show that when the setup is "highly over-parametrized", stochastic descent methods have similar minimax optimality and implicit-regularization properties. This observation gives some insight into why deep networks exhibit such powerful generalization abilities. They also allow one to construct new algorithms that can incorporate explicit regularization.
10:30-11:30
Talk by Mahdi Soltanolkotabi, University of Southern California
Towards Stronger Foundations for AI: Overparameterized Learning Beyond the Lazy Regime
Abstract: Despite wide empirical success, many of the most commonly used learning approaches lack a clear mathematical foundation and often rely on poorly understood heuristics. Even when theoretical guarantees do exist they are often too crude and/or pessimistic to explain their success in practical regimes of operation or serve as a guiding principle for practitioners. Furthermore, in many scenarios such as those arising in scientific applications they require significant resources (compute, data, etc.) to work reliably.
This talk takes a step towards building a stronger theoretical foundation for such nonconvex learning. In particular, I will focus on demystifying the generalization and feature learning capability of modern overparameterized learning where the parameters of the learning model (e.g. neural network) exceed the size of the training data. We will discuss our results for a variety of problems including assymetric low rank reconstruction from a few measurements, linear neural networks and improved results for one-hidden layer neural nets with ReLU activations. Our result is based on an intriguing spectral bias phenomena for gradient descent, that puts the iterations on a particular trajectory towards solutions that are not only globally optimal but also generalize well. Notably this analysis overcomes a major theoretical bottleneck in the existing literature and goes beyond the “lazy” training regime which requires unrealistic hyperparameter choices (e.g. very small step sizes, large initialization or wide models).
Bio: Mahdi Soltanolkotabi is an associate professor and an Andrew and Erna Viterbi Early Career Chair in ECE, CS, & ISE at the University of Southern California and the director of the Center for AI Foundations for the Sciences (AIF4S). Prior to joining USC, he completed his PhD in electrical engineering at Stanford in 2014. He was a postdoctoral researcher in the EECS department at UC Berkeley during the 2014-2015 academic year. His research focuses on developing the mathematical foundations of modern data science via characterizing the behavior and pitfalls of contemporary nonconvex learning and optimization algorithms with applications in deep learning, large scale distributed training, federated learning, computational imaging, and AI for scientific applications. Mahdi is the recipient of the Information Theory Society Best Paper Award, Packard Fellowship in Science and Engineering, a Sloan Research Fellowship in mathematics, an NSF Career award, an Airforce Office of Research Young Investigator award (AFOSR-YIP), the Viterbi school of engineering junior faculty research award, and faculty research awards from Google and Amazon.
11:30-13:00
Lunch (not included in the program)
13:00-14:00
Talk by Bruno Lourier, EPFL
Gaussian Universality in learning at high-dimensions
Abstract: While classical in many theoretical settings, the assumption of Gaussian covariates is often perceived as a strong limitation in the analysis of learning problems. Recent results, however, suggest the opposite: in high-dimensions Gaussian covariates encompass a larger universality class than one might have initially expected. In this talk, I will discuss this recent line of work and its (current) limitations.
Bio: Bruno Loureiro is currently a research scientist at the "Information, Learning and Physics" (IdePHICS) laboratory at EPFL, working on the crossroads between Machine Learning and Statistical Physics. Before moving to EPFL, he was a postdoctoral researcher at the Institut de Physique Théorique (IPhT) in Paris, and received his PhD from the University of Cambridge. He is interested in Bayesian inference, theoretical machine learning and high-dimensional statistics more broadly. His research aims at understanding how data structure, optimisation algorithms and architecture design come together in successful learning.
14:00-15:00
Talk by Arnulf Jentzen (online), Chinese University of Hong Kong, Shenzhen and University of Münster
Overcoming the curse of dimensionality: from nonlinear Monte Carlo to the training of neural networks
Abstract: Partial differential equations (PDEs) are among the most universal tools used in modelling problems in nature and man-made complex systems. Nearly all traditional approximation algorithms for PDEs in the literature suffer from the so-called ”curse of dimensionality” in the sense that the number of required computational operations of the approximation algorithm to achieve a given approximation accuracy grows exponentially in the dimension of the considered PDE. With such algorithms it is impossible to approximatively compute solutions of high-dimensional PDEs even when the fastest currently available computers are used. In the case of linear parabolic PDEs and approximations at a fixed space-time point, the curse of dimensionality can be overcome by means of Monte Carlo approximation algorithms and the Feynman-Kac formula. In the first part of this talk we present an efficient machine learning algorithm to approximate solutions of high-dimensional PDE and we also prove that suitable deep neural network approximations do indeed overcome the curse of dimensionality in the case of a general class of semilinear parabolic PDEs. In the second part of the talk we present some recent mathematical results on the training of neural networks.
References:
[1] Becker, S., Jentzen, A., Müller, M. S., and von Wurstemberger, P., Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing. arXiv:2202.02717 (2022), 70 pp.
[2] Hutzenthaler, M., Jentzen, A., Kruse, T., and Nguyen, T. A., A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations. Partial Differ. Equ. Appl. 1 (2020), no. 2, Paper no. 10, 34 pp.
[3] Jentzen, A. and Riekert, A., On the existence of global minima and convergence analyses for gradient descent methods in the training of deep neural networks. J. Mach. Learn. 1 (2022), no. 2, 141–246.
Bio: Arnulf Jentzen (*November 1983) is appointed as a presidential chair professor at the School of Data Science and the Shenzhen Research Institute of Big Data at the Chinese University of Hong Kong, Shenzhen (since 2021) and as a full professor at the Faculty of Mathematics and Computer Science at the University of Münster (since 2019). In 2004 he started his undergraduate studies in mathematics (minor field of study: computer science) at Goethe University Frankfurt in Germany, in 2007 he received his diploma degree at this university, and in 2009 he completed his PhD in mathematics at this university. The core research topics of his research group are machine learning approximation algorithms, computational stochastics, numerical analysis for high dimensional partial differential equations (PDEs), stochastic analysis, and computational finance. He is particularly interested in deep learning based algorithms for high dimensional approximation problems and different kinds of differential equations. At the moment he serves as an associate, division, or managing editor for the Annals of Applied Probability (AAP, since 2019), for Communications in Computational Phyiscs (CiCP, since 2021s), for Communications in Mathematical Sciences (CMS, since 2015), for Discrete and Continuous Dynamical Systems Series B (DCDS-B, since 2018), for the Journal of Applied Mathematics and Physics (ZAMP, since 2016), for the Journal of Complexity (JoC, since 2016), for the Journal of Machine Learning (JML, since 2021), for the Journal of Mathematical Analysis and Applications (JMAA, since 2014), for the SIAM / ASA Journal on Uncertainty Quantification (JUQ, since 2020), for the SIAM Journal on Scientific Computing (SISC, since 2020), for the SIAM Journal on Numerical Analysis (SINUM, since 2016), and for Partial Differential Equations and Applications (PDEA, since 2019). His research activities have been recognized by several scientific prizes. In particular, in 2020 he has been awarded the Felix Klein Prize from European Mathematical Society (EMS) and in 2022 he has been awarded the Joseph F. Traub Prize for Achievement in Information-Based Complexity. Further details on the activities of his research group can be found at the webpage http://www.ajentzen.de.
15:00-16:00
Coffee break
16:00-16:30
Christopher Zach, Chalmers
Bilevel programs meet deep learning: error propagation as a meta-learning task
Abstract: Today error back-propagation is the main tool underneath most training methods for artificial neural networks. Unfortunately, it is also an algorithm that is highly implausible to be realized in our brains, and there is consequently an active search for biologically more plausible alternatives to back-propagation. One set of such neuroscience-inspired methods builds on the old idea of contrastive Hebbian learning, where network activations are not computed in a forward pass but inferred by minimizing an appropriate "network energy." We show how various recent flavors of contrastive Hebbian learning naturally emerge by extending suitable bilevel minimization programs to deeper nesting levels. Thus, in meta-learning terminology, the task of inferring network activations corresponds to a training loss, and parameter learning itself maps to a meta-objective. As by-products we obtain a slightly generalized variant of back-propagation as a special case, and we are further able to work around differentiability issues of quantized neural networks.
16:00-16:30
Fredrik Hellström, Chalmers
Non-vacuous generalization guarantees for deep learning via information-theoretic bounds
Abstract: TBA
August 30
10:30-11:30
Talk by Christos Thrampoulidis, University of British Columbia
Finding Structures in Large Models: Imbalance Trouble
Abstract: What are the unique structural properties of models learned deep nets? Is there an implicit bias towards solutions of a certain geometry? How does this vary across training instances, architectures, and data? Towards answering these questions, the recently discovered Neural Collapse phenomenon formalizes simple geometric properties of the learned embeddings and of the classifiers, which appear to be “cross-situational invariant” across architectures and different balanced classification datasets.
But what happens when classes are imbalanced? Is there a (ideally equally simple) description of the geometry that is invariant across class-imbalanced datasets? By characterizing the global optima of an unconstrained-features model, we formalize a new geometry that remains invariant across different imbalance levels. Importantly, it, too, has a simple description despite the asymmetries imposed by data imbalances on the geometric properties of different classes.
Overall, we show that it is possible to extend the scope of the neural-collapse phenomenon to a richer class of geometric structures. We also motivate further investigations into the impact of class imbalances on the implicit bias of first-order methods and into the potential connections between such geometry structures and generalization.
11:30- 13:00
Lunch (not included in the program)
13:00-14:00
Diyora Salimova (online): Albert-Ludwigs-University of Freiburg
Deep neural network approximations for PDEs
Abstract: Most of the numerical approximation methods for PDEs in the scientific literature suffer from the so-called curse of dimensionality (CoD) in the sense that the number of computational operations and/or the number of parameters employed in the corresponding approximation scheme grows exponentially in the PDE dimension and/or the reciprocal of the desired approximation precision. Recently, certain deep learning-based approximation methods for PDEs have been proposed and various numerical simulations for such methods suggest that deep neural network (DNN) approximations might have the capacity to indeed overcome the CoD in the sense that the number of real parameters used to describe the approximating DNNs grows at most polynomially in both the PDE dimension and the reciprocal of the prescribed approximation accuracy. In this talk, we show that solutions of suitable Kolmogorov PDEs can be approximated by DNNs without the CoD.
Bio: Since the beginning of this year I have been a Junior Professor of Applied Mathematics at the University of Freiburg. My research interests lie within mathematical analysis of machine learning as well as stochastic and numerical analysis of SDEs and PDEs. Prior to joining the University of Freiburg I was a Foundations of Data Science postdoctoral Fellow at ETH Zurich. I obtained my Phd in Mathematics in 2019 from ETH Zurich under the supervision of Arnulf Jentzen. I also have an MSc and a BSc in Mathematics.
14:00-15:00
Vahid Tarokh, Duke University (TBA)
15:00-16:00
Coffee Break
16:00-17:00
Alexandros G. Dimakis, University of Texas at Austin
Deep Generative models and Unsupervised methods for Inverse problems
Abstract: Modern deep generative models like GANs, VAEs, Score-based models and Diffusions are demonstrating excellent performance in representing high-dimensional distributions, especially for images. We will show how they can be used to solve inverse problems like denoising, filling missing data, and recovery from linear projections. We generalize compressed sensing theory beyond sparsity, extending Restricted Isometries to sets created by deep generative models. Our recent results include establishing theoretical results for Langevin sampling from full-dimensional generative models, generative models for MRI reconstruction and fairness guarantees for inverse problems.
Bio: Alex Dimakis is a UT Austin Professor and the co-director of the National AI Institute on the Foundations of Machine Learning (IFML). He received his Ph.D. from UC Berkeley and the Diploma degree from NTU in Athens, Greece. He has published more than 150 papers and received several awards including the James Massey Award, NSF Career, a Google research award, the UC Berkeley Eli Jury dissertation award, and several best paper awards. He served as an Associate Editor for several journals, as an Area Chair for major Machine Learning conferences (NeurIPS, ICML, AAAI) and as the chair of the Technical Committee for MLSys 2021. His research interests include information theory and machine learning. He is an IEEE Fellow for contributions to distributed coding and learning.
17:00-18:00
Maxim Raginsky, University of Illinois at Urbana-Champaign
Learning Recurrent Neural Net Models of Nonlinear Systems
Abstract: In this talk, based on joint work with Joshua Hanson (UIUC) and Eduardo Sontag (Northeastern), I will discuss the following learning problem: Given sample pairs of input and output signals generated by an unknown nonlinear system (which is not assumed to be causal or time-invariant), we wish to find a continuous-time recurrent neural net that approximately reproduces the underlying i/o behavior with high confidence. Leveraging earlier work concerned with matching output derivatives up to a given finite order, we reformulate the learning problem in familiar system-theoretic language and derive quantitative guarantees on the sup-norm risk of the learned model in terms of the number of neurons, the sample size, the number of derivatives being matched, and the regularity properties of the inputs, the outputs, and the unknown i/o map.
Bio: Maxim Raginsky received the B.S. and M.S. degrees in 2000 and the Ph.D. degree in 2002 from Northwestern University, all in Electrical Engineering. He has held research positions with Northwestern, the University of Illinois at Urbana-Champaign (where he was a Beckman Foundation Fellow from 2004 to 2007), and Duke University. In 2012, he has returned to the UIUC, where he is currently a Professor and William L. Everitt Fellow with the Department of Electrical and Computer Engineering and the Coordinated Science Laboratory. He also holds a courtesy appointment with the Department of Computer Science. Prof. Raginsky's interests cover probability and stochastic processes, deterministic and stochastic control, machine learning, optimization, and information theory. Much of his recent research is motivated by fundamental questions in modeling, learning, and simulation of nonlinear dynamical systems, with applications to advanced electronics, autonomy, and artificial intelligence.
August 31
9:30-11:30
Q/A panel Discussion
Send your questions to (with a suggested speaker to respond): qa.chair.toml@gmail.com