Resources

Book / 2019

Michael Kearns and Aaron Roth incorporate philosophy, ethics, politics, and economics in their discussion of the tradeoff between accurate prediction and fairness constraints put on those predictions.

This workshop held on Dec 11, 2020 holds resources for AI researchers and marginalized communities to discuss and reflect on AI-fueled inequity and co-create dreams and tactics of how to work toward Resistance AI.

ICQCM's mission is to advance the presence of scholars of color among those using data science methodologies, and challenge researchers to use those methods in ways that can dismantle the structural barriers to enable human flourishing for underrepresented communities, professionals, and young people.

The Distributed AI Research Institute is a space for independent, community-rooted AI research free from Big Tech’s pervasive influence, launched by Timnit Gebru.

This resource is run by Carnegie Mellon University and the Data Science for Social Good Foundation. This page includes key questions key questions that data scientists should ask as they pursue their projects. Framing questions include: Who will be affected by our work? How we are ensuring that by doing ‘good’ for one group, we are not inadvertently harming another?

Recorded webinar April 14, 2021 with Alex Hanna.

Problems of algorithmic bias are often framed in terms of lack of representative data or formal fairness optimization constraints to be applied to automated decision-making systems. However, these discussions sidestep deeper issues with data used in AI, including problematic categorizations and the extractive logics of crowdwork and data mining. In this talk, Dr. Hanna makes two interventions: first by reframing of data as a form of infrastructure, and as such, implicating politics and power in the construction of datasets; and secondly discussing the development of a research program around the genealogy of datasets used in machine learning and AI systems.

Sample tools from IG&H (Netherlands-based company) that incorporate ethics into the design of data projects.

Housed in Princeton University’s Department of African American Studies, the IDA B. WELLS Just Data Lab brings together students, educators, activists, and artists to develop a critical and creative approach to data conception, production, and circulation. The aim is to rethink and retool the relationship between stories and statistics, power and technology, data and justice. Founding director of lab is Ruha Benjamin.

Since economists (and many other disciplines!) have been thinking about how to deal with biases like selection bias, omitted variable biases, etc. there is a budding literature on the intersection of machine learning and causal inference.

Developed by Dario Sansome at University of Exeter

Preprint / Conference paper 2019

"Model cards are short documents accompanying trained machine learning models that provide benchmarked evaluation in a variety of conditions, such as across different cultural, demographic, or phenotypic groups ... and intersectional groups ... that are relevant to the intended application domains. Model cards also disclose the context in which models are intended to be used, details of the performance evaluation procedures, and other relevant information. "

Resources from UCLA scholar Safiya Noble

A critique of the way search engine algorithms are designed to reflect the negative bias of their creators, particularly at the expense of women of color. They offer a skewed perception of the people's thoughts and beliefs, which is often mistaken as reality.

Podcast with Su Lin Blodgett

(March 2021)

How do we define bias? Is all bias the same? Is it possible to eliminate bias completely in our AI systems? Should we even try?

Preprint / Research paper / 2020

Critiquing the recognition of bias in existing NLP research. Provides three recommendations for "how researchers and practitioners conducting work analyzing "bias" in NLP systems might avoid the pitfalls presented".

NOVA video / April 14, 2021

Safiya Noble and Latanya Sweeney shed light on the hidden biases in search engine technology and specifically discuss how it marginalizes women of color.

"This course examines the interactions of technology and power, in particular, how technology enforces and extends both state and privatized forms of power."

Position paper / Jan 2020

Lewis, Jason Edward, ed. 2020. Indigenous Protocol and Artificial Intelligence Position Paper. Honolulu, Hawaiʻi: The Initiative for Indigenous Futures and the Canadian Institute for Advanced Research (CIFAR).

Article/Report / April 2021

"Automatic gender recognition systems can't recognize the transgender people exist and is incompatible with self expression. Algorithmic transparency is needed to enforce a ban on this technology."

Book by Catherine D'Ignazio & Lauren F. Klein / 2020 / MIT Press

Open Access

Today, data science is a form of power. It has been used to expose injustice, improve health outcomes, and topple governments. But it has also been used to discriminate, police, and surveil. This potential for good, on the one hand, and harm, on the other, makes it essential to ask: Data science by whom? Data science for whom? Data science with whose interests in mind? The narratives around big data and data science are overwhelmingly white, male, and techno-heroic.

The Manifest-No is a declaration of refusal and commitment. It refuses harmful data regimes and commits to new data futures.

Gender Shades is a preliminary excavation of the inadvertent negligence that will cripple the age of automation and further exacerbate inequality if left to fester. The deeper we dig, the more remnants of bias we will find in our technology. We cannot afford to look away this time, because the stakes are simply too high. We risk losing the gains made with the civil rights movement and women's movement under the false assumption of machine neutrality. Automated systems are not inherently neutral. They reflect the priorities, preferences, and prejudices—the coded gaze—of those who have the power to mold artificial intelligence.

Joy Buolamwini, Lead Author

Book / 2017

Mathematician and data scientist Cathy O’Neil reveals the mathematical models being used today are unregulated and uncontestable, even when they’re wrong. Most troubling, they reinforce discrimination—propping up the lucky, punishing the downtrodden, and undermining our democracy in the process.

Preprint / 2021

Machine learning (ML) currently exerts an outsized influence on the world, increasingly affecting communities and institutional practices. It is therefore critical that we question vague conceptions of the field as value-neutral or universally beneficial, and investigate what specific values the field is advancing. In this paper, we present a rigorous examination of the values of the field by quantitatively and qualitatively analyzing 100 highly cited ML papers published at premier ML conferences, ICML and NeurIPS. We annotate key features of papers which reveal their values: how they justify their choice of project, which aspects they uplift, their consideration of potential negative consequences, and their institutional affiliations and funding sources. We find that societal needs are typically very loosely connected to the choice of project, if mentioned at all, and that consideration of negative consequences is extremely rare.