References
General
Practical Uncertainty Estimation and Out-of-Distribution Robustness in Deep Learning. NeurIPS 2020 Tutorial.
A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification. 2021.
Bayesian Deep Learning and a Probabilistic Perspective of Model Construction. ICML 2020 Tutorial.
Probability Calibration Workshop. PyData 2020.
Uncertainty Estimation and Bayesian Neural Networks. PyData 2018.
Symposium on Conformal and Probabilistic Prediction with Applications. COPA conference 2012-2022.
Workshop on Distribution-Free Uncertainty Quantification. DFUQ @ ICML 2021-2022.
Uncertainty & Robustness in Deep Learning Workshop. UDL @ ICML 2022.
Probability calibration
Desai & Durrett "Calibration of Pre-trained Transformers". EMNLP 2020.
Kuleshov & Liang "Calibrated Structured Prediction". NeurIPS 2015.
Kumar & Sarawagi. "Calibration of Encoder-Decoder Models for Neural Machine Translation". 2019.
Guo et al. "On Calibration of Modern Neural Networks". ICML 2017.
Naeini et al. "Obtaining Well Calibrated Probabilities Using Bayesian Binning". AAAI 2015.
Kumar et. al. "Verified Uncertainty Calibration". NeurIPS 2020.
Kadavath et al. "Language Models (Mostly) Know What They Know". 2022.
Lin et al. "Teaching Models to Express Their Uncertainty in Words". 2022.
Liu et al. "BRIO: Bringing Order to Abstractive Summarization". ACL 2022
Bayesian approaches
Houlsby et al. "Bayesian Active Learning for Classification and Preference Learning." 2011.
Xiao & Wang "Quantifying Uncertainties in Natural Language Processing Tasks". AAAI 2019.
Conformal Prediction
Angelopoulos et al. “Uncertainty Sets for Image Classifiers using Conformal Prediction”. ICLR 2021
Maltoudoglou et al. “BERT-based conformal predictor for sentiment analysis”. CPPA 2020
Dey et al. “Conformal prediction for text infilling and part-of-speech prediction”. 2021
Schuster et al. “Consistent Accelerated Inference via Confident Adaptive Transformers”. EMNLP 2021
Angelopoulos et al. “Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control“. 2021
Schuster et al. “Confident Adaptive Language Modeling”. NeurIPS 2022
Tibshirani et al. “Conformal Prediction Under Covariate Shift”. 2020
Card et al. "Deep Weighted Averaging Classifiers". FAT* 2019
Selective prediction and out-of-domain detection
El-Yaniv and Weiner. "On the foundations of noise-free selective classification." JMLR 2010.
Kamath, Jia, and Liang. “Selective Question Answering under Domain Shift.” ACL 2020.
Hendrycks, Mazeika, and Dietterich. “Deep Anomaly Detection with Outlier Exposure.” ICLR 2019.
Arora, Huang, and He. “Types of Out-of-Distribution Texts and How to Detect Them.” EMNLP 2021.