ICLR2023 workshop on
Pitfalls of limited data and computation for Trustworthy ML
May 5, 2023, Kigali, Rwanda
Due to the impressive performance of ML algorithms, they are increasingly used in a wide range of applications that impact our daily lives. These include sensitive domains like healthcare, banking, social services, autonomous transportation, social media, advertisement, etc. However, ML algorithms that are deployed in the real world are restricted by a multitude of computational and statistical limitations. Often ignored in the ML research pipeline, these restrictions include
Statistical limitations: lack of available data, limited availability of high-quality labelled data, and lack of data from different domains of interest
Computational limitations: lack of high-speed hardware, lack of high memory hardware, extreme constraints on the computation time of ML algorithms during training or inference, and lack of hardware (e.g. hardware that cannot exploit sparsity) that is suitable for specific kinds of computations
It is necessary to understand the impact of such limitations on the performance of ML algorithms. As these algorithms are increasingly used for high-stakes decision-making in socially impactful domains, their trustworthiness is becoming an increasingly relevant design factor to consider. In recent years, several issues with the trustworthiness of ML algorithms have been identified:
Privacy: Leaking private information about the training data.
Fairness: Incurring disparate impact on sensitive subpopulations.
Miscalibration: Giving a false sense of reliability through miscalibrated predictions.
Reproducibility: Inconsistency across multiple runs of the ML pipeline.
Distribution shift: Sensitivity to natural and adversarial test distribution shifts.
Robustness: Vulnerability to noise in the training data.
Safety and Reliability: Causing issues in the safety of resulting applications.
Explainability and Interpretability: Identifying factors leading to predictions.
Auditing and Certifying ML systems: Challenges of audit and certification under limited data and compute.
In this workshop, we want to invite theoretical and empirical researchers to come together and discuss barriers to trustworthy ML and algorithms that overcome them. To enable this, we will solicit submissions that address questions such as (but not limited to) the following:
How does having less data or poor-quality data affect the trustworthiness of ML algorithms? Can these problems be mitigated with new algorithmic techniques (e.g. SSL, new DNN models, active learning)?
How do computational limitations impact the trustworthiness of ML algorithms? What are some natural statistical tasks that exhibit fundamental trade-offs between computational efficiency (runtime, memory, etc.) and trustworthiness (fairness, privacy, robustness)? Are these trade-offs also observed in practice?
Do these limitations result in trade-offs between different aspects of trustworthiness? If yes, how can they be averted with relaxations or new algorithmic techniques?
Accepted papers: https://openreview.net/group?id=ICLR.cc/2023/Workshop/Trustworthy_ML
Please contact us at trustml-unlimited-workshop-2023@googlegroups.com with any questions regarding the workshop.
Invited Speakers
Title: Impacts of Data Scarcity on Groups and Harnessing LLMs for Solution
Abstract: In this talk, I address the challenges posed by underspecification and data scarcity in machine learning, focusing on the varying impacts on different groups. I review prior methods like selective classification for addressing these challenges and discuss their limitations in modern machine learning.
To overcome these issues, I highlight the necessity of empowering individuals to create data based on their unique concepts. However, data generation has its own challenges, as it is difficult to create data for a concept without introducing shortcuts or interference with the original data or other concepts. To overcome these obstacles, I introduce CoDev, a novel framework for the collaborative development of NLP models. CoDev enables individuals to collaborate with AI and each other to generate data in a controlled manner that respects the integrity of existing concepts and original data. I conclude the talk by discussing the inherent limitations of data that persist even in the presence of infinite data.
Bio: Fereshte Khani is a senior applied scientist in the Office of Applied Research at Microsoft. Before joining Microsoft, she received her Ph.D. at Stanford under the supervision of Percy Liang. In her research, Fereshte tries to understand how one can investigate and mitigate discrimination of ML models, and how to study the feedback loops created by ML models. Her current work focuses on proposing novel methodology for fair, robust, and reliable machine learning.
Title: Practical poisoning of machine learning models
Abstract: Deep learning models are often trained on distributed, webscale datasets crawled from the internet. However, due to their size, these datasets are necessarily uncurated. This opens the possibility for a "poisoning attack" that would allow an adversary to modify the behavior of a model. With our attack I could have poisoned the training dataset for anyone who has used LAION-400M (or other popular datasets) in the last six months. Our attack is trivial: I bought expired domains corressponding to URLs in popular image datasets. This gave us control over 0.01% of each of these datasets. In this talk I discuss how the attack works, the consequences of this attack, and potential defenses. More broadly, we hope machine learning researchers will study other simple but practical attacks on the machine learning pipeline.
Bio: Nicholas Carlini is a research scientist at Google Brain working at the intersection of machine learning and computer security. His most recent line of work studies properties of neural networks from an adversarial perspective. He received his Ph.D. from UC Berkeley in 2018, and his B.A. in computer science and mathematics (also from UC Berkeley) in 2013. He is broadly interested in developing attacks on machine learning systems; his work develops attacks demonstrating security and privacy risks of these systems.
Title: Towards neural networks robust to distribution shifts
Abstract: Despite their success, the performance of neural networks has been shown to be brittle to mismatch between train and test distributions. Previous works have hypothesized that this brittleness is caused because deep networks rely only on simple features of the input (such as background or texture of images) to make decisions, while completely ignoring complex features. Surprisingly, we find that the features learnt by network’s backbone are sufficient for out of distribution generalization, however, the final classifier layer trained using ERM does not use these features optimally for the same. We posit two reasons for this:
1.dominance of non-robust features
2.replication of simple features, leading to over-dependence of the max-margin classifier on these.
We empirically validate these hypotheses on semi-synthetic and real-world datasets. We also draw connections with the line of work studying simplicity bias of neural nets. We then propose two methods to deal with both of these phenomena, and show gains of upto 1.5% over the state-of-the-art on DomainBed - a standard and large-scale benchmark for domain generalization.
Based on joint works with Anshul Nasery, Sravanti Addepalli, R. Venkatesh Babu and Prateek Jain.
Bio: Praneeth Netrapalli is a research scientist at Google Research India, Bengaluru, adjunct professor at TIFR, Mumbai and faculty associate of ICTS, Bengaluru. Prior to this, he was a researcher at Microsoft Research India for 4.5 years and did his postdoc at Microsoft Research New England in Cambridge, MA. Praneeth has been awarded the 2021 INSA Medal for Young Scientists and the 2019 IEEE Signal Processing Society Best Paper Award for his work. His current research interests are broadly in designing reliable and robust machine learning (ML) algorithms, with a focus on solving problems from sciences as well as enabling positive social outcomes.
Title: How (not) to Model an Adversary
Abstract: Statistical learning (and theory) traditionally relies on training and test data being generated by the same process, an assumption that rarely holds in practice. Conditions of data-generation might change over time, or agents might (strategically or adversarially) respond to a published predictor aiming for a specific outcome for their manipulated instance. Developing methods for adversarial robustness has received a lot of attention in recent years, and both practical tools and theoretical guarantees developed. In this talk, I will focus on the learning theoretic treatment of these scenarios and survey how different modeling assumptions can lead to drastically different conclusions. I will argue that for robustness we should aim for minimal assumptions on how an adversary might act, and present recent results on a variety of relaxations of learning with standard adversarial (or strategic) robustness.
Bio: Ruth Urner is an Assistant Professor at York University. She is also faculty affiliate at Toronto's Vector Institute. Previous to that she was a senior research scientist at the Max Planck Institute for Intelligent Systems in Tübingen, Germany, and a postdoctoral fellow at Carnegie Mellon's Machine Learning department as well as at Georgia Tech. Her research develops mathematical tools and frameworks for analyzing the possibilities and limitations of automated learning, with a focus on semi-supervised, active and transfer learning. Currently she is particularly interested in developing formal foundations for topics relating to societal impacts of machine learning, such as human interpretability and algorithmic fairness.
Title: What Neural Networks Memorize and Why
Abstract: Deep learning algorithms tend to fit the entire training dataset thereby memorizing even noisy labels. In addition, complex models have been shown to memorize entire input examples, including seemingly irrelevant information (social security numbers from text, for example). This puzzling propensity to memorize seemingly useless data
is not explained by existing theories of machine learning. We provide simple conceptual explanations and theoretical models demonstrating that memorization of labels and training examples is necessary for achieving close-to-optimal generalization error when learning from long-tailed data distributions. This holds despite the fact that most of that information is ultimately irrelevant to the learning task at hand. Our results allow us to quantify the cost of limiting memorization in learning and explain the disparate effects that privacy and model compression have on different subpopulations. Finally, we demonstrate the utility of memorization and support our explanation empirically. These results rely on a new technique for efficiently estimating memorization and influence of training data points.
Bio: Vitaly Feldman is a research scientist at Apple ML Research working on foundations of machine learning and privacy-preserving data analysis. His recent research interests include tools for analysis of generalisation, distributed privacy-preserving learning, privacy-preserving optimization, and adaptive data analysis. His work on understanding of memorization in learning was recognized by the 2021 Caspar Bowden Award for Outstanding Research in Privacy Enhancing Technologies and his research on foundations of adaptive data analysis was featured in CACM Research Highlights and Science. His works were also recognized by COLT Best Student Paper Award in 2005 and 2013 (student co-authored) and by the IBM Research Best Paper Award in 2014, 2015 and 2016.
