(currently maintained by Dinghuai Zhang and Irina Rish)
Surveys:
Domain Generalization: A Survey
Papers:
Invariant Risk Minimization: An Information Theoretic View (blog)
Leon Bottou's talk: Learning Representations Using Causal Inference (Workshop on Theory of Deep Learning: Where next?)
Linear unit-tests for invariance discovery several linear settings as testbed for IRM, REx, etc...
Invariant Risk Minimization Games (ICML2020) game-theoretic reformulation of IRM (Kartik Ahuja's talk on IRM Games paper)
Shortcut Learning in Deep Neural Networks (Nature machine intelligence) A relevant survey
Invariant Rationalization interesting formulation, invariance + mutual information
Domain Extrapolation via Regret Minimization similar to IRM, replace the loss in a regret form
Out-of-Distribution Generalization via Risk Extrapolation (REx) (slides) (ICML2021) generalization of IRM, propose MM-REx and V-REx (use variance among losses from domains as penalty)
Learning Robust Representations with Score Invariant Learning modification of REx, replace loss with || \nabla_{\theta} loss ||
Risk Variance Penalization: From Distributional Robustness to Causality generalization of REx, replace variance with standard deviation....
Generalization and Invariances in the Presence of Unobserved Confounding seems exactly same as (while 4 months later than) REx...
Out-of-Distribution Generalization with Maximal Invariant Predictor modification to V-REx, replace "loss in one domain" with "gradient of loss in that domain"
Learning Causal Models Online IRM setting + continual learning
Self-training Avoids Using Spurious Features Under Domain Shift theoretically prove self-training can prevent from using spurious feature in linear setting
Gradient Starvation: A Learning Proclivity in Neural Networks analyze sigmoid binary classification with Legendre dual, propose a new regularization ||y_hat||.
Improving out-of-distribution generalization via multi-task self-supervised pretraining
Learning explanations that are hard to vary (ILC) (ICLR2021) Invariant Learning Consistency (ILC) criterion, (approx) AND-masking of gradients to only keep directions consistent (across domains)
In Search of Lost Domain Generalization (ICLR2021)
The Risks of Invariant Risk Minimization (ICLR2021) When the number of environments is small, IRM will fall
Understanding the Failure Modes of Out-of-Distribution Generalization (ICLR2021)
REPRESENTATION LEARNING VIA INVARIANT CAUSAL MECHANISMS (ICLR2021) improve contrastive learning with style-augmentation
SYSTEMATIC GENERALISATION WITH GROUP INVARIANT PREDICTIONS (ICLR2021)
Empirical or Invariant Risk Minimization? A Sample Complexity Perspective (ICLR2021) Theoretically calculate sample complexity of IRM (which is similar to ERM)
ICLR2021 rejected submissions:
OUT-OF-DISTRIBUTION PREDICTION WITH INVARIANT RISK MINIMIZATION: THE LIMITATION AND AN EFFECTIVE FIX irm + transfer learning loss trick (like mmd loss)
INVARIANT CAUSAL REPRESENTATION LEARNING
Domain-Free Adversarial Splitting for Domain Generalization IRM + environment unaware setting + link to fairness
OUT-OF-DISTRIBUTION GENERALIZATION ANALYSIS VIA INFLUENCE FUNCTION analysis via influence function
LEARNING ROBUST MODELS USING THE PRINCIPLE OF INDEPENDENT CAUSAL MECHANISMS causal + normalizing flow + HSIC for independence
Does Invariant Risk Minimization Capture Invariance? (AISTATS2021 oral) fairly good paper, analyzing when will IRMv1 fail (to get the solution of IRM), and when will IRM itself fail
Linear Regression Games: Convergence Guarantees to Approximate Out-of-Distribution Solutions (AISTATS2021)
Meta-Learned Invariant Risk Minimization MAML + REx ...
Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization punish the dot product of gradients from different models to enforce diversity + OOD model selection
Environment Inference for Invariant Learning (ICML2021) demographic unaware setting (and some discussion related to fairness)
Predict then Interpolate: A Simple Algorithm to Learn Stable Classifiers (ICML2021)
Heterogeneous Risk Minimization (ICML2021)
Just Train Twice: Improving Group Robustness without Training Group Information (ICML2021) train once, then re-train while up-weighting the data that previously has large loss
OoD-Bench: Benchmarking and Understanding Out-of-Distribution Generalization Datasets and Algorithms
Towards a Theoretical Framework of Out-of-Distribution Generalization adopt a conditional DA like term to bound OOD error
An Information-theoretic Approach to Distribution Shifts
An Empirical Investigation of Domain Generalization with Empirical Risk Minimizers (NeurIPS2021) Ben David's bound is not practical for predicting generalization, exploring other kinds of measure
Causality & ML
Causality for Machine Learning
Invariance, Causality and Robustness
Causal inference using invariant prediction: identification and confidence intervals ICP (invariant causal prediction) paper
Invariant Causal Prediction for Nonlinear Models nonlinear ICP
Invariant Causal Prediction for Block MDPs ICP in MDP settings
Invariant Models for Causal Transfer Learning causal transfer paper
Anchor regression: heterogeneous data meets causality
Learning Independent Causal Mechanisms