Abstracts

We study offline reinforcement learning (RL), which seeks to learn a good policy based on a fixed, pre-collected dataset. A fundamental challenge behind this task is the distributional shift due to the dataset lacking sufficient exploration, especially under function approximation. To tackle this issue, a bi-level structured policy optimization algorithm is proposed that models a hierarchical interaction between the policy (upper level) and the value function (lower level). The lower level focuses on constructing a confidence set of value estimates that maintain sufficiently small weighted average Bellman errors while controlling uncertainty arising from distribution mismatch. Subsequently, at the upper level, the policy aims to maximize a conservative value estimate from the confidence set formed at the lower level. This novel formulation preserves the maximum flexibility of the implicitly induced exploratory data distribution, enabling the power of model extrapolation. In practice, it can be solved through a computationally efficient, penalized adversarial estimation procedure. The theoretical regret guarantees do not rely on any data-coverage and completeness-type assumptions, only requiring realizability. These guarantees also demonstrate that the learned policy represents the best effort among all policies, as no other policies can outperform it. 

ML models in critical applications like healthcare and finance often rely on incomplete, noisy, or biased data, where ground truth is unrecoverable. Conventional data cleaning falls short, leading to models that appear fair and accurate during development but fail in deployment. We first discuss an approach for learning fair models by leveraging background knowledge from the data collection process and external knowledge about the domain to enable learning fair models that are robust to incomplete data.

We then present a novel framework that systematically propagates uncertainty throughout the training pipeline to enhance model robustness when data is incomplete or uncertain. Our approach integrates Possible Worlds Semantics from database theory and utilizes abstract interpretation techniques to analyze robustness by propagating variations caused by incomplete and uncertain data into models and their decisions. This framework ensures a principled way to quantify and address uncertainty, enabling reliable predictions under varying data conditions.

Much of the research on causal discovery methods focuses on methods able to handle increasingly complex systems, whether in terms of number of variables, density of connections, complexity of parametric functions, and more. In contrast, this talk will focus on causal discovery when we have challenges due to complicated measurements. Specifically, I focus on two different types of challenges. First, we might not be able to measure at the same timescale as the causal connections; for example, fMRI measurements are much slower than the brain’s timescale, or economic measures are typically released less frequently than economic activity occurs. I will describe several causal discovery algorithms that we have developed for this situation, demonstrating both theoretical and real-world performance. I also demonstrate that we can sometimes benefit by measuring even more slowly, contra the standard advice that “faster measurement is better measurement.” Second, we might not be able to measure all of the relevant variables at the same time; for example, a healthcare organization and financial institution might each measure variables that interest them, but we cannot (for privacy reasons) integrate those into a single dataset. I describe causal discovery methods that can extract information about the global causal structure from these partial, overlapping datasets, and show that the methods can yield novel scientific insights on real-world data.

The causal revolution has spurred interest in understanding complex relationships across various fields. Most existing methods aim to discover causal relationships among variables within a complex, large-scale system. However, in practice, only a small number of variables are relevant to the outcomes of interest. As a result, causal estimation using the full causal representation, especially with limited data, could lead to many falsely discovered, spurious variables that are highly correlated with but have no causal impact on the target outcome. We propose learning a class of necessary and sufficient causal representation that only contain causally relevant variables, utilizing probabilities of causation. The framework is further extended to natural language processing models to disentangle the 'black box' by identifying true rationales when two or more snippets are highly inter-correlated, thus contributing similarly to prediction accuracy. We leverage two causal desiderata, non-spuriousness and efficiency, establishing their theoretical identification as the main component of learning necessary and sufficient in language models. The superior performance of our proposed methods is demonstrated in real-world reviews and medical datasets through extensive experiments.

Causality lies at the heart of scientific inquiry, serving as the fundamental basis for understanding the interactions among variables in physical systems. Despite its central role, current methods for causal inference face significant challenges due to the presence of nonlinear dependencies, stochastic and deterministic interactions, self-causation, mediator, confounder, and collider effects, and contamination from unobserved, exogenous factors, to name a few. While there are methods that can effectively address some of these challenges, no single approach has been successful in integrating all these aspects. Here, we tackle these challenges with SURD: Synergistic-Unique-Redundant Decomposition of causality. SURD quantifies causality as the increments of redundant, unique, and synergistic information gained about future events based on available information from past observations. The formulation is non-intrusive and requires only pairs of past and future events, facilitating its application in both computational and experimental investigations, even when samples are scarce. We benchmark SURD against existing methods in scenarios that pose significant challenges in causal inference. These include synchronization in logistic maps, the Rössler-Lorenz system, the Lotka-Volterra prey-predator model, the Moran effect model, and energy cascade in turbulence, among others. Our findings demonstrate that SURD offers a more reliable quantification of causality compared to state-of-the-art methods for causal inference.