Tel Aviv University Department of Statistics and Operations Research
Seminar takes place on Tuesdays 10:30-11:30 at Schriber 309 (TAU map ) unless noted otherwise .
For questions or to subscribe to emailing list: danielnevo at tauex.tau.ac.il
Scroll to bottom for a shareable calender
Special Summer Seminars (press for details):
Title: Speciation without Isolation in the Bateson-Dobzhansky-Muller Model and (a taste of) Dispersion Estimation in RNA Sequencing
Abstract:
I will begin with a short overview of our work on (over)-dispersion estimation in RNA sequencing, followed by a more detailed presentation of the main topic: Speciation without isolation in the Bateson-Dobzhansky-Muller (BDM) model. This is a model which describes a basic and well established scheme for the process of speciation through mutation accumulation, which has central importance in modern speciation research. Previous studies add assumptions to this model, which are meant to represent realistic scenarios, but also to facilitate theoretical analysis or accelerate simulation convergence. In this work we present a taxonomy of these models and assumptions and discuss their realism. We investigate speciation dynamics in the BDM model without further assumptions, and quantify the effect of additional assumptions on the results. Our main simulation scenario assumes a circular geographic structure, with localized preferential choice of mates, representing a simplistic ”modern human” species. Our simulations demonstrate that in the BDM model several species appear and tend to stabilize in a steady state. With additional assumptions results vary dramatically, from almost immediate and extreme speciation to no speciation in our covered time scales. We conclude that seemingly harmless assumptions can have a major effect on the results of speciation studies, and that the basic BDM should be adopted as the benchmark for examining the effect and realism of such assumptions.
This seminar presents results from my PhD thesis, done under the supervision of Prof. Saharon Rosset
Title: Smooth Contextual Bandits
Abstract:
Contextual bandit problems model the inherent tradeoff between exploration and exploitation in personalized decision making in marketing, healthcare, revenue management, and more. Specifically, the tradeoff is characterized by the optimal growth rate of the regret in cumulative rewards compared to the optimal policy. Naturally, the optimal rate should depend on how complex the underlying supervised learning problem is, namely how much can observing reward in one context tell us about mean rewards in another context. Curiously, this obvious-seeming relationship is obscured in current theory that separately studies the easy, fully-extrapolatable case and hard, super-local case. To characterize the relationship more precisely I study a nonparametric contextual bandit problem where expected reward functions are β-smooth (roughly meaning β-times differentiable). I will show how this interpolates between the two extremes previously studied in isolation: non-differentiable-response bandits (β ≤ 1), where rate-optimal regret is achieved by decomposing the problem into non-contextual bandits, and parametric-response bandits (β = ∞), where rate-optimal regret is often achievable without any exploration at all. We develop a novel algorithm that works for any given smoothness setting by operating neither fully locally nor fully globally. We prove its regret is rate-optimal, thereby characterizing the optimal regret rate and revealing a fuller picture of the crucial interplay between complexity and regret in dynamic decision making. Time permitting, I will also discuss how to construct valid confidence intervals from data collected by contextual bandits, a crucial challenge in the enterprise to replace randomized trials with adaptive experiments in applied fields from biostatistics to development economics.
This talk is based on the following papers:
Smooth Contextual Bandits: Bridging the Parametric and Non-differentiable Regret Regimes
https://pubsonline.informs.org/doi/abs/10.1287/opre.2021.2237
Post-Contextual-Bandit Inference
https://papers.nips.cc/paper/2021/hash/eff3058117fd4cf4d4c3af12e273a40f-Abstract.html
Title: Clusters disagreements between training and testing data.
Abstract:
We introduce a disagreement problem that may be an issue when classification is implemented on estimated classes of the population. A disagreement problem denotes the case in which a sample fails to cover a specific class that a testing observation belongs to. Or in other words, the training and testing data do not agree on the number of classes in the population. These disagreement problems may occur due to various reasons such as sampling errors, selection bias, or emerging classes of the population. Once the disagreement problem occurs, a testing observation will be misclassified, because a classification rule based on the sample cannot capture a class not observed in the training data (sample). To overcome such issues, we suggest a two-stage classification method that can ameliorate a disagreement problem in classification. Our proposed method tests whether a testing observation is sampled from the observed or possibly unobserved class, then classifies it based on the test result. We suggest a test for identification of the disagreement problem and demonstrate the performance of the two-stage classification via numerical studies. (This is a joint work with Jung Wun Lee)
Second Semester (press for details):
Title: Baroque Simplicity: Simplicity Bias in Overparameterized Machine Learning
Abstract:
A thorough theoretical understanding of the surprising generalization ability of deep networks (and other overparameterized models) is still lacking. Here we demonstrate that simplicity bias is a major phenomenon to be reckoned with in overparameterized machine learning. In addition to explaining the outcome of simplicity bias, we also study its source: following concrete rigorous examples, we argue that
(i) simplicity bias easily explains generalization in overparameterized learning models such as neural networks;
(ii) simplicity bias and excellent generalization are optimizer-independent, as our example shows, and although the optimizer affects training, it is not the driving force behind simplicity bias;
(iii) simplicity bias in pre-training models, and subsequent posteriors, is universal and stems from the subtle fact that uniformly-at-random constructed priors are not uniformly-at-random sampled; and
(iv) in neural network models, the biasing mechanism in wide (and shallow) networks is different from the biasing mechanism in deep (and narrow) networks.
Thus, while (ii) and (iv) take issue with current theory that focuses on wide networks and stochastic gradient descent for explaining generalization, another path is suggested by (i) and (iii).
Title: Distribution-Free Bayesian multivariate predictive inference
Abstract:
I will introduce a Bayesian multivariate predictive inference framework. The basis for this framework is a hierarchical Bayesian model, that is a mixture of finite Polya trees corresponding to multiple dyadic partitions of the unit cube. Given a sample of observations from an unknown multivariate distribution, the posterior predictive distribution is used to model and generate future observations from the unknown distribution. I will present an algorithm for constructing conformal prediction sets, that provide finite sample probability assurances for future observations, with our Bayesian model.
I will illustrate the use of the methodology on two examples from Itay Manes’s dissertation, forecasting weekly Dengue virus cases and estimating average causal effect from observational studies, and discuss its application for data imputation and differential privacy.
Title: Cheating with Models
Abstract:
Beliefs and decisions are often based on confronting models with data. What is the largest "fake" correlation that a misspecified model can generate, even when it passes an elementary misspecification test? We study an "analyst" who fits a model, represented by a directed acyclic graph, to an objective (multivariate) Gaussian distribution. We characterize the maximal estimated pairwise correlation for generic Gaussian objective distributions, subject to the constraint that the estimated model preserves the marginal distribution of any individual variable. As the number of model variables grows, the estimated correlation can become arbitrarily close to one, regardless of the objective correlation.
Title: Is bias necessary for learning with more parameters than examples?
Abstract:
Modern machine learning practices tend to interpolate highly expressive models that are susceptible to overfitting with far more free parameters than examples. This is in contrast with common wisdom that requires the number of free parameters to scale with the data. To resolve this alleged contradiction it is often claimed that one has to look at the learning algorithm, in particular its inductive bias. In this talk, we will revisit this paradigm in one of the most well-studied and well-understood models for theoretical machine learning: Stochastic Convex Optimization (SCO).
We begin by new results that highlight the role of the optimization algorithm in learning. These results demonstrate that in SCO much like in practice learning can be done in an overparameterized regime (i.e. more parameters than examples). Specifically, we show separation results between the generalization performance of stochastic gradient descent (SGD) and of gradient descent (GD), or regularized GD. We next discuss the inductive bias of Stochastic Gradient Descent (SGD) as well as its stability, and we ask if the implicit bias or stability accounts for the success of SGD to generalize. We provide several constructions that point out significant difficulties in providing a comprehensive explanation of an algorithm's generalization performance by solely arguing about its implicit regularization or stability.
On the one hand, these results demonstrate the importance of the optimization algorithm in generalization. On the other hand, they also hint that the reason or cause for the different performances may not necessarily be explained or understood via investigations of the algorithm's bias.
Based on joint works with: Idan Amir, Assaf Dauber, Meir Feder, Tomer Koren, Yishay Mansour, Uri Shermann, Nathan Srebro.
No Seminar
Title: Data-driven medicine in practice
Abstract:
In recent years, healthcare is characterized by an explosion of demand and an overload of information. These trends make it more and more challenging to provide the most accurate and up-to-date treatment for each patient. However, owing to its high-quality medical data, Israel is well situated to turn these challenges into opportunities.
This talk will focus on how data from an integrated healthcare system (Clalit) can be utilized to promote predictive, proactive, and personalized care. The development process of several clinical prediction models will be presented, along with their integration into the into the point-of-care.
Title: A Statistical Analysis of Automatic Evaluation for Summarization
Abstract:
Text summarization in NLP is the process of summarizing the information in large texts for quicker consumption. The quality of a summarization evaluation metric is quantified by calculating the correlation between its scores and human annotations across a large number of summaries. Currently, it is not clear how precise these correlation estimates are, nor whether differences between two metrics’ correlations reflect a true difference or if it is due to random chance. In this talk, I will address these two problems by proposing methods for calculating confidence intervals and running hypothesis tests for correlations measured in summarization. After evaluating which of the proposed methods is most appropriate for summarization through two simulation experiments, I will analyze the results of applying these methods to several different automatic evaluation metrics across three sets of human annotations. In this research, we find that the confidence intervals are rather wide, demonstrating high uncertainty in how reliable automatic metrics truly are. Further, although many metrics fail to show statistical improvements over ROUGE, two recent works, QAEval and BERTScore, do in some evaluation settings. This work is published at TACL 2021: https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00417/107833/A-Statistical-Analysis-of-Summarization-Evaluation.
In the second part of this talk, I will present an ongoing study that identifies two ways in which the definition of the system-level correlation is inconsistent with how metrics are used to evaluate summarization systems in practice and propose changes to rectify this disconnect. The results from these analyses point to the need for future research to focus on developing more consistent and reliable human evaluations of summaries.
This research was done in collaboration with Daniel Deutsch, a Ph.D. student from the Cognitive Computation Group at the Department of Computer and Information Science, University of Pennsylvania.
Title: TBA
Abstract:
TBA
Title: Statistical and mathematical modeling of COVID vaccination in Israel
Abstract:
The focus of the lecture will be issues around planning and estimating the effectiveness of the different vaccination campaigns. Israel was one of the first countries to administer mass vaccination. Consequently, it was among the first countries to experience substantial breakthrough infections due to the waning of vaccine-induced immunity, which led to a resurgence of the epidemic. In response, Israel launched a booster campaign to mitigate the outbreak, and was the first country to do so. Israel’s success in curtailing the Delta resurgence while imposing only mild non-pharmaceutical interventions influenced the decision of many countries to initiate a booster campaign. By constructing a detailed mathematical model and calibrating it to the Israeli data, we extended the understanding of the impact of the booster campaign from the individual to the population level. We used the calibrated model to explore counterfactual scenarios in which the booster vaccination campaign was altered by changing the eligibility criteria or the start time of the campaign and to assess the direct and indirect effects in the different scenarios. The results point to the vast benefits of vaccinating younger age groups not at high risk of developing severe disease but who play an important role in transmission. We further show that when the epidemic grows exponentially, the success of the booster campaign is highly sensitive to the timing of its initiation. I will discuss the differences between the Delta (fourth wave) and the Omicron (fifth wave) and the logic behind the decision to boost only the high-risk individuals during the Omicron wave.
Title: The Trimmed Lasso: Sparse Recovery Guarantees and Practical Optimization
Abstract:
Consider the sparse approximation or best subset selection problem: Given a vector y and a matrix A, find a k-sparse vector x that minimizes the residual ||Ax-y||. This sparse linear regression problem, and related variants, plays a key role in high dimensional statistics, compressed sensing, and more. In this talk we focus on the trimmed lasso penalty, defined as the L_1 norm of x minus the L_1 norm of its top k entries in absolute value. We advocate using this penalty by deriving sparse recovery guarantees for it, and by presenting a practical approach to optimize it. Our computational approach is based on the generalized soft-min penalty, a smooth surrogate that takes into account all possible k-sparse patterns. We derive a polynomial time algorithm to compute it, which in turn yields a novel method for the best subset selection problem. Numerical simulations illustrate its competitive performance compared to current state of the art.
Joint work with Tal Amir and Ronen Basri.
Title: Statistical Approaches for Assessing and Enhancing Replicability
Abstract:
The scientific method has been undermined over the past two decades for lack of replicability, especially being a cross disciplinary crisis. Great efforts have been invested to study and enhance results’ replicability, starting with the study design, its implementation, through the development of analysis tools, and finally offering replicability metrics. Although the problems are the same across fields of science, the specific challenges and techniques are field-specific and therefore requiring special solutions per field.
I shall discuss the replicability problem in two fields. The first is experimentation with mouse models, which is a central strategy in preclinical testing of pharmacological treatments and understanding the effect of genes on behaviour. This problem led to the need to develop a scheme for improved unbiased estimation of interaction variance in ANOVA models with unbalanced group sizes. The second is meta-analyses of clinical-trials, where the concern is about assessing replicability of intervention effects.
Title: Regression analysis with censored covariates in the presence of a cured fraction
Abstract:
Many health surveys collect data in a cross-sectional way and record only a current value of a Patient Reported Outcome under study. Some information on the patient's disease history is usually collected as well. However, any such cross-sectional study encounters a problem that at the time of data collection some of the important events, which are related to the studied outcome and affect it, had happened for some of the survey participants, but not for others. In addition, the incompleteness (or censoring) in this ``life-changing'' time-to-event covariate might be complicated by the presence of a cured fraction, i.e., those patients who do belong to the target population but will never experience this event.
An example of such data is the data obtained from the Web-based Adult Perthes Survey, which was launched in order to collect data on physical, mental and social functioning of patients who had a rare Perthes disease in childhood. A total hip replacement (THR) surgery in adulthood is a life-changing event for those Perthes patients who need it, but it might not be needed for some of the Perthes patients (a cured fraction).
In this paper we aim to estimate trajectories of physical functioning of Perthes patients in adulthood, and associations between their physical functioning and their Perthes history, including a THR and the age-at-THR.
There are quite few available approaches for cross-sectionally measured outcomes with censored covariates, but none of them account for a cured fraction. We propose a two-stage approach based on a pseudo-likelihood, and assess its finite sample performance in simulations, both under correctly specified model and under model misspecification. We derive its asymptotic properties. Finally, we apply our approach to the data on the long term outcomes of Perthes, collected in the Adult Perthes Survey.
Title: On the number of maximum random vectors: new results for some old problems
Abstract:
Consider $n$ i.i.d. real-valued random vectors of size $k$ having iid coordinates with a general distribution function $F$. A vector is a maximum if and only if there is no other vector in the sample which weakly dominates it in all coordinates. The distribution of $M_{k,n}$, the number of maximums, is of interest in multi-objective optimization and other fields, and several combinatorial and asymptotic results are known in the literature.
Here, we establish new results, focusing mainly on the case where both $k$ and $n$ grow to infinity.
First, we show a phase transition for the normalized expectation $E[M_{k,n}]/n$: If $k\equiv k_n$ is growing at a slower (faster) rate than a certain factor of $\log(n)$, then $E[M_{k,n}]/n \rightarrow 0$ (resp. $E[M_{k,n}]/n\rightarrow1$) as $n\to\infty$. Furthermore, the factor is fully characterized as a functional of $F$. We also study the effect of $F$ on $E[M_{k,n}]/n$, showing that while $E[M_{k,n}]/n$ may be highly affected by the choice of $F$, the phase transition is the same for all distribution functions up to a constant factor.
Second, we show a new combinatorial formula for the variance of $M_{k,n}$. We then present a new correlation inequality for vectors in the unit hypercube $[0,1]^ k$. This inequality is used to prove the asymptotic independence of the events of two different vectors being a maximum. Finally, these results are used to establish a central limit theorem for certain linear functionals of the number of maximums among some subsets of the $n$ vectors, as $k,n \to \infty$.
Joint work with Royi Jacobovic
Title: Screening human IVF embryos for polygenic diseases: potential and limitations
Abstract:
Polygenic risk scores (PRSs) are offered commercially in the USA to screen IVF embryos for genetic liability to adult diseases, despite an incomplete understanding of the expected outcomes and social and ethical concerns. We used statistical genetics modeling to predict the expected reduction in complex disease risk when screening embryos for a single disease. We show that selecting the embryo with the lowest PRS can lead to substantial relative risk reductions under realistic, though best-case, settings. I will survey the impact of several parameters on the expected risk reduction, including the accuracy of the PRS, the number of available embryos, the disease prevalence, and the parental disease status. I will also discuss practical limitations and barriers to implementation.
First Semester (press for details):
Title: Simultaneous inference for the false discovery proportion
Abstract:
Closed testing can be used to obtain simultaneous confidence bounds for the false discovery proportion. In this talk we examine the special case of closed testing with Simes local tests, by highlighting its connections with Hommel and Benjamini-Hochberg methods. We also touch upon the case of interval hypotheses, for which simultaneous two-sided confidence limits are possible. The proposed procedure is illustrated in the context of fMRI data.
Department meeting
Title: Multiple testing of partial conjunction null hypotheses
Abstract:
The partial conjunction null hypothesis is tested in order to discover a signal that is present in multiple studies. We propose methods for multiple testing of partial conjunction null hypotheses which make use of conditional p-values based on combination test statistics. Specific examples comprise the Fisher combination function and the Stouffer combination function. The conditional validity of the corresponding p-values is proved for certain classes of one-parametric statistical models, including one-parameter natural exponential families. The standard approach of carrying out a multiple test procedure on the (unconditional) partial conjunction p-values can be extremely conservative. We suggest alleviating this conservativeness, by eliminating many of the conservative partial conjunction p-values prior to the application of a multiple test procedure. This leads to the following two step procedure: first, select the set with partial conjunction p-values below a selection threshold; second, within the selected set only, apply a family-wise error rate or false discovery rate controlling procedure on the conditional partial conjunction p-values. By means of computer simulations and real data analyses, we compare the proposed methodology with other recent approaches.
This is joint work with Thorsten Dickhaus and Anh-Tuan Hoang
Title: Causal inference for semi-competing risks data
Abstract:
An emerging challenge for time-to-event data is studying semi-competing risks, namely when two event times are of interest: a non-terminal event (e.g. Alzheimer's disease diagnosis) time, and a terminal event (e.g. death) time. The non-terminal event is observed only if it precedes the terminal event, which may occur before or after the non-terminal event. Studying treatment or intervention effects on the dual event times is complicated because for some units, the non-terminal event may occur under one treatment value but not under the other. Until recently, existing approaches generally disregarded the time-to-event nature of both outcomes. More recent research focused on principal strata effects within time-varying populations coupled with Bayesian estimation. In this paper, we present alternative estimands, based on a single stratification of the population, corresponding to the scientific questions of interest. We present a novel assumption utilizing the time-to-event nature of the data, that is generally more flexible than the often-invoked monotonicity assumption. Our new assumption enables partial identifiability of causal effects of interest. We further present a frailty-based sensitivity analysis approach, and give conditions under which full identification is possible. We present non-parametric and semi-parametric estimation methods under right censoring. We illustrate the utility of our approach in a study of the causal effects of having APOE e4 allele on late-onset Alzheimer's disease and death.
Joint work with Malka Gorfine
Title: COVID-19 during 2021 in Israel: waning immunity and the booster dose effect
Abstract:
In December 2020, Israel began a mass vaccination campaign against COVID-19 administering the Pfizer BNT162b2 vaccine, which led to a sharp curtailing of the outbreak. After a period with almost no SARS-CoV-2 infections, a resurgent COVID-19 outbreak began in mid-June 2021. On July 30, 2021, a third (booster) dose of the vaccine was approved in Israel for individuals 60 years or older who had received the second dose at least five months previously. The booster vaccination campaign in Israel was gradually expanded to younger age groups who received a second dose five or more months earlier. Using Israel's national database, we quantify the extent of waning immunity of two doses against the Delta variant, and estimate the difference in the rate of confirmed infection and severe COVID-19 between recipients of only two doses of vaccine and those also receiving a third booster dose. In this talk, I present the data, the methods used to analyze them, and the results that indicate the effectiveness of the booster dose for all age groups.
The work is joint with Micha Mandel, Yinon Bar-On, Omri Bodenheimer, Laurence Freedman, Sharon Alroy-Preis, Nachman Ash, Amit Huppert, and Ron Milo.
Title: Detecting changes between correlation matrices due to single-variable perturbations.
Abstract:
I’ll discuss the challenges in detecting local differences between correlation matrix populations. When the correlation matrices are large,
mass-univariate testing each correlation coefficient individually may be underpowered to detect differences.
We therefore propose a model that assumes single-variables are perturbed, thereby pooling together the effects along columns (and rows) of the correlation matrix.
Inference methods for the model are tailored for comparing samples of correlation matrices, in that they account for both the inherent variability in correlation matrices and for the variation between matrices in each sample. In simulations, our model shows a substantial increase in power compared to the mass univariate approaches.
As a test case, we analyze correlation matrices of resting state functional-MRI (RS-fMRI) in patients with a rare neurological condition -
transient global amnesia (TGA) and healthy controls. In this dataset, our model identifies substantially decreased synchronization in several brain regions within the patient population, which could not be detected using previous methods without prior-knowledge.
This is joint work with Itamar Faran, Michael Peer and Shahar Arzi.
Title: DOPE: D-Optimal Pooling Experimental design with application for SARS-CoV-2 screening
Abstract:
I will show how Bayesian D-optimality can be utilized to considerably increase SARS-CoV-2 testing throughout and accuracy. I will start by presenting a Bayesian formulation of (pooled) SARS-CoV-2 testing, where D-optimal testing schemes are defined as maximizing mutual information between parameters and data. Next, I will present numerical techniques for finding such D-optimal testing schemes, utilizing vanilla and Markov-Chain Monte-Carlo. These efforts will culminate in DOPE (D-Optimal Pooling Experimental design) --- a novel testing scheme for SARS-CoV-2. Simulation results demonstrating DOPE's superiority to several benchmarks and suggest several future research directions will conclude my talk.
Title: The perils of data preprocessing prior to cross-validation
Abstract:
Regression and classification methods are typically evaluated by cross-validation: repeatedly splitting the data into a training set and a validation set, learning a predictive model on the training set, then averaging its loss on the validation set. Under the i.i.d. assumption, this should give an unbiased and consistent estimator for the risk of the trained model. However, in practice, many data sets go through various stages of preprocessing, such as rescaling, dimensionality reduction, and outlier removal. Such ``unsupervised'' preprocessing procedures, that do not involve the responses or class labels, are often considered harlmess. However, they introduce a subtle leakage of information from the validation set to the trained model. This breaks the assumptions of cross-validation, potentially leading to biased risk estimates and sub-optimal model selection. In this talk, we make the case that this subtle error should receive more attention since it is prevalent in scientific research, potentially harmful, and typically easy to fix. We will explain where the bias is coming from, how to eliminate it, and walk through several toy examples, demonstrating an intricate dependency between the parameters of the problem and the resulting bias due to preprocessing.
Joint work with Saharon Rosset.
Title: Seeing the Unseen - a Generalized Scheme for Missing Mass Estimation
Abstract:
Consider a finite sample from an unknown distribution over a countable alphabet. The missing mass refers to the total probability of symbols that do not appear in the sample. Estimating the missing mass is a basic problem in statistics and related fields, which dates back to the early work of Laplace, and the more recent seminal contribution of Good and Turing. In this work we introduce a generalized framework for missing mass estimation. Our proposed framework provides novel risk bounds and improves upon currently known estimation schemes. Importantly, it is easy to apply and does not require additional modeling assumptions. This makes it a favorable choice for many practical applications.
Title: The partial linear model (PLM) from semiparametric to modern ramifications.
Abstract:
Engle, Granger, Rice, and Weiss (1986) suggested the partially linear model to deal with regression, which has both linear and nonparametric components. Its modern equivalent, inference in the ultra high dimensional regression, was analyzed by Zhang and Zhang (2014) and others. We consider different variations on this theme, including inference without assuming compatibility, design of experiments with unlabeled data, single-index models, and regression discontinuity designs.
Add calender for non-google calenders: link to ics file
Past years:
Talks from before the 2021/2022 academic year can be found at this link
