Communication Across Communities in Machine Learning Research and Practice

sedl.facct2022@gmail.com

@SEDL_workshop

SEDL @ FAccT 2022

Join us here!

21-24 June 2022

  1. SEDL returns in 2022 at FAccT conference in the form of a CRAFT unconference entitled Communication Across Communities in Machine Learning Research and Practice. In Communication Across Communities we will critically examine what it means to build participatory methods in contemporary machine learning.

  2. Engagement with our CRAFT will begin in advance in a Discord channel for preliminary discussions, in particular early prototyping of the collaborative lexicon.

Goals of SEDL 3.0

The first two editions of "Science meets/and Engineering in Deep Learning" focused on the research communities: 1. analysing the fragmentation in ML research and enhancing communication; 2. reflecting on the impact that ML has in science and engineering applications. In this third iteration we want to expand further the discussion including society into the equation.

Crossing disciplinary boundaries, we propose a collaborative CRAFT session focused on extant and idealized communication practices between machine learning’s broad and seemingly disparate set of stakeholders, including researchers and practitioners, creators and artists, social scientists and legal scholars, policy-makers and citizens, users and bystanders. Folding these voices into conversation with one another, we aim to highlight commonalities, conflict and moments of creative tension.

Designed with participation in mind, our whole CRAFT session will be a collective effort, broken out into three stand-alone yet interrelated parts. Part 1 will consider and critique how art communicates concepts surrounding ML to broader audiences. Part 2 will explore shared and divergent vocabulary through real-time collaborative development of a machine learning lexicon. Finally, part 3 will present an informal and open-floor panel on the discordance between standardized evaluation and contextual sensitivity.



The Unconference

June 21 (hybrid) and 24 (online) 2022

SEDL 3.0 is divided into three parts:

Artistic narratives as a form of communication

Art served scientific purposes in the attempt to explain the events of life. With the increasing complexity of society, artworks, specifically paintings and orally transmitted plays, also became educational tools, teaching the illiterate population about religious doctrine. This process wasn't of course only meant to educate the public, it also played a major role in sharing and maintaining specific ideas and concepts, usually those officially recognised by the ruling classes throughout history. As a form of resistance towards this phenomenon, creative work became a way to oppose such ideologies too. Thus Art became an effective way to expose the public to often controversial new ideas and perspectives, often opposing or deviating from what’s accepted and acceptable.


In this part, we would like to discuss the potential influences of artistic communications as a tool of cross-disciplinary knowledge transfer. We would also like to discuss the impact of machine learning in the art community and how this cross-disciplinary communications could create potential harms and benefits. We aim to bring together artists, including computational artists who use machine learning as a tool of creation and communication with researchers who work on the intersection of art and society. Discussion points include

  1. Care and ethical considerations in ML-oriented creation and artistic storytelling

  2. Emphasizing marginalized narratives and representation issues in ML

  3. Critique of narratives around large scale ML techniques potentially accelerating harms


We believe that a great degree of care and research should be put on this way of communication to prevent further marginalizing of the communities that are already marginalized. We would like to learn how art serves as a point of interdisciplinary communications and what we, as the people at the intersection of Theoretical and Applied ML, FAccT, and Social Science, could implement in our communities.


  1. Building an inclusive lexicon for ML
    Communication across communities through various forms of narratives necessarily rely on shared vocabularies whose content may diverge depending on which community uses it. Concepts in machine learning via this shared vocabulary are being used by many communities, however, diverging meanings arise as a result of disciplinary differences. In this part, we aim to build an inclusive lexicon of machine learning. We believe that building a shared vocabulary will help in cross disciplinary communication and will complement the other parts of this proposal.

    We plan to work towards an inclusive lexicon of machine learning with particular attention to the social impact of AI (starting with a description of what AI is). The concepts will not only be descriptive but also prescriptive, especially for concepts with potential social impact. The lexicon will be built communally and the preliminary work will be done before the event. During the 90 minute discussion the key topics will include:

    1. How to build an inclusive lexicon focusing on diverging vocabulary

    2. Usefulness for descriptions and the need for prescriptive entries in the lexicon

    3. Baseline definitions, building a map of machine learning, and its editorial structure

    4. Clarifying fuzzy concepts with prioritizing their social impact

We will include a discussion of related references and find ways to make this effort public.


  1. ML evaluations, standards and critiques
    Earlier parts of the proposed session explore how individual context shapes interpretation of our shared vocabulary, and the remarkable ability of art(ists) to communicate with people across contexts. However, a crucial perspective remains absent from our discussion, that of real people whose lives are affected by our research, systems, and practices.


This final part will be a semi-structured panel-style discussion that will consider practical problems surrounding participatory ML evaluation. From data collection, to algorithm design and implementation, to application and deployment in society, ML requires careful consideration of its impacts. Core questions discussed in this breakout will include how (often dynamic) values are encoded in our evaluation practices; the participation and role of broad and inclusive stakeholder groups; and the conflict between “standardized” evaluation practices and sensitivity to a system’s eventual context and use.


Additionally, this part will engage with participatory design literature from the human-computer interaction (HCI) and social computing community. Such scholarship historically has been of focus within these communities. Especially as it interfaces with scholarship, it explores the intersection of gender and participatory design, participatory design and health and participatory design for community focused research. Drawing from community-focused design practices that have been explored within HCI, we hope to explore the ways in which participatory design can support critical research within the FAccT community.


This part will engage participants in interrogating if it is possible to develop standard practices for evaluation, how this might be approached, and how different contexts require different solutions. Prior to the event, participants will be invited to complete an online survey whose results will frame a starting point for discussion.


Program

June 21 - 14:30-16:00 Seoul time (07:30-09:00 CET, 01:30-03:00 ET, 22:30-00:00 PT)
ML evaluations, standards and critiques [Hybrid]


14:30-15:15 : discussion panel. Panellists: Katherine Heller, Xiao Liu, Stephanie Chan, Shalaleh Rismani, Jesse Dodge. Moderator: Margaret Mithcell.

15:15-16:00 : open discussion.


June 21 - 16:30-18:00 Seoul time (09:30-11:00 CET, 03:00-05:00 ET,00:30-02:00 PT)
Building an inclusive lexicon for ML [Hybrid]


16:30-16:50 : recorded talks from Andi Peng and Savannah Thais

16:50-17:15 : in-person talks and discussion with Nyalleng Moorosi and Skyler Wang.

17:15-18:00 : hackathon.


June 24 - 04:30-06:00 Seoul time (June 23 - 21:30-23:00 CET, 15:30-17:00 ET, 12:30-14:00 PT)
Artistic narratives as a form of communication [Online]


4:30-6:00 : discussion panel. Panellists: Andy Smart, Sofia Crespo, Mark Leibert, Alexander Reben, Amy Alexander, Suzanne Livingston. Moderators: Ramya Malur Srinivasan and Negar Rostamzadeh.


The program can still be subject to small changes before its final version.

FAQ

  1. Is there a difference between SEDL & established workshops focusing on ‘AI for Science’ or ‘Responsible AI’?
    At first sight, the themes we focus on may appear to overlap with existing, established workshops. However, our focus is not on the examples themselves, rather on how we conduct our research and communicate our results.

Organizers

Adriana Romero
META AI Research

Jessica Forde
Brown University

Levent Sagun
META AI Research

Samuel Bell
University of Cambridge

Seyi Olojo
UC Berkeley