Optimal Mechanisms for Demand Response: An Indifference Set Approach

Renewable energy production, like solar or wind, is not controllable, but consumers often have flexibility in their energy use. Demand-response programs aim to shift consumption to align with renewable energy availability. For example, a home energy management system (HEMS) can adjust heating or cooling schedules to match renewable production while keeping indoor temperatures within user-defined bounds. We show that while dynamic pricing does not always lead to optimal demand response, it becomes asymptotically optimal as the number of consumers grows. We develop an efficient algorithm for determining optimal prices by querying HEMS consumption schedules under different price signals.

In preparation for submission to Management Science

Mohammad Mehrabi, Omer Karaduman, Stefan Wager

More details

Off-Policy Evaluation in Markov Decision Processes under Weak Distributional Overlap

Doubly robust methods offer promising results for off-policy evaluation in Markov decision processes (MDPs), with efficient large-sample convergence and modular implementation using standard reinforcement learning techniques. However, they often assume strong distributional overlap, which may not hold in unbounded state spaces. We introduce truncated doubly robust (TDR) estimators that perform well under weaker overlap conditions. Our approach recovers large-sample behavior when the distribution ratio is square-integrable, and even when it's not, TDR remains consistent, achieving a minimax rate of convergence over a class of MDPs defined by mixing conditions.

In preparation for submission to Operations Research

Mohammad Mehrabi, Stefan Wager

More details

Online Debiasing for Adaptively Collected High-Dimensional Data with Applications to Time Series Analysis

We propose online debiasing, a novel approach for reliable statistical inference for settings with high-dimensional adaptively collected data. In particular, in two concrete contexts (i) time series analysis and (ii) batched data collection, we demonstrate that online debiasing optimally debiases the LASSO estimate when the underlying parameter has sparsity of certain order.

Journal of the American Statistical Association, 2023 Apr 3; 118(542):1126-39.

Yash Deshpande, Adel Javanmard, Mohammad Mehrabi (alphabetical order)

More details

Adversarial robustness for latent models: Revisiting the robust-standard accuracies tradeoff

It has been observed that recent adversarial training methods, while improving model robustness, often reduce standard test accuracy. This has led us to explore the tradeoff between standard accuracy (generalization) and robust accuracy (robust generalization). In this paper, we revisit this tradeoff for latent models and argue that it can be mitigated when data exhibit a low-dimensional structure. Our theory shows that such a structure enables models to achieve near-optimal performance in both standard and robust accuracy.

Operations Research, 2024 May; 72(3):1016-30.

Adel Javanmard, Mohammad Mehrabi (alphabetical order)

More details

GRASP: A Goodness-of-fit Test for Classification Learning

Classifier performance is often measured by average accuracy, which can miss the model’s fit to the conditional law of labels given features (Y|X). To address this, we propose GRASP, a novel test for assessing goodness-of-fit without parametric assumptions, treating Y|X as a black-box model. GRASP is distribution-free and works in finite samples. We also introduce model-X GRASP, which enhances power by using known feature distributions.

Journal of the Royal Statistical Society Series B, 2024 Feb; 86(1):215-45.

Adel Javanmard, Mohammad Mehrabi (alphabetical order)

More details

Error-Correction for Sparse Support Recovery Algorithms

We propose a low complexity procedure that is designed to be used generically on top of any sub-optimal baseline support recovery algorithm to improve its statistical accuracy. This can be achieved at the cost of a relatively small computational overhead. Interestingly, adding this module to orthogonal matching pursuit (OMP) yields a support recovery procedure that is more accurate and significantly faster than basis pursuit (BP).

IEEE Transactions on Information Theory. 2022 Jul 4;68(11):7396-409

Mohammad Mehrabi, Aslan Tchamkerten

More details

Pearson Chi-Squared Conditional Randomization (PCR) Test

We introduce the Pearson Chi-squared Conditional Randomization (PCR) test, a novel approach to conditional independence testing in the model-X framework. Building on the strengths of the Conditional Randomization Test (CRT), the PCR test enhances statistical power by using the Pearson Chi-squared test to score data subsets, allowing flexible analysis across varying group sizes. This method addresses key limitations of the CRT, including the need for a large number of randomizations, by leveraging the i.i.d. property of data samples to generate high-resolution p-values with fewer randomizations.

Submitted to Annals of Statistics

Adel Javanmard, Mohammad Mehrabi (alphabetical order)

More details

Conference Proceedings

A Model-free Closeness-of-influence Test for Features in Supervised Learning

Mohammad Mehrabi, and Ryan A. Rossi

International Conference on Machine Learning (ICML), 2023

Download

Fundamental Tradeoffs in Distributionally Adversarial Training

Mohammad Mehrabi, Adel Javanmard, Ryan A. Rossi, Anup Rao, and Tung Mai

International Conference on Machine Learning (ICML), 2021

Spotlight paper, acceptance rate: 3.5%

Download

Error-Correction for Sparse Support Recovery Algorithms

Mohammad Mehrabi, Aslan Tchamkerten

IEEE International Symposium on Information Theory (ISIT), 2021

Download

Bounds on the Approximation Power of Feedforward Neural Networks

Mohammad Mehrabi, Aslan Tchamkerten

International Conference on Machine Learning (ICML), 2018

Download