Bridging the Gap from AI Ethics Research to Practice

Kathy Baxter is Architect of Ethical AI Practice at Salesforce with over 20 years of experience in the tech industry. She develops research-informed best practice to educate Salesforce employees, customers, and the industry on the development of ethical AI. You can read about her research at Einstein.ai/ethics. She received her MS in Engineering Psychology and a BS degree in Applied Psychology from the Georgia Institute of Technology. The second edition of her book, "Understanding your users," was published in May 2015.

Yoav Schlesinger is Principal of Ethical AI Practice at Salesforce. He helps instantiate, embed and scale industry-leading best practices for the responsible development, use and deployment of AI. Prior to joining this team, Yoav worked at Omidyar Network where he led the Responsible Computer Science Challenge and helped develop EthicalOS, a risk mitigation toolkit for product managers. Before that, he leveraged his undergraduate studies in Religious Studies and Political Science as a leader of mission-driven, social impact organizations.

Joaquin Quinonero Candela, Director of Engineering, Facebook (with Isabel Kloumann, Research Science Manager): The presenter will focus on fairness in Facebook’s approach to fairness in product development, with an emphasis on fairness being a process that considers the holistic impact and interaction between people and systems, rather than a specific state or property of data or an algorithm. Facebook's approach will be illustrated through a case study of fairness in a content moderation system.

Krishnaram Kenthapadi, Principal Scientist, Machine Learning Services, Amazon AI [formerly LinkedIn]: The presenter will share LinkedIn’s fairness-aware reranking for talent search. The key lesson from LinkedIn's experience will be highlighted: that building consensus and achieving collaboration across key stakeholders (such as product, legal, PR, engineering, and AI/ML teams) is a prerequisite for successful adoption of fairness-aware approaches in practice.

Lewis Baker, Director of Data Science, Pymetrics: Pymetrics builds predictive models of job success that are tailored to a specific roles at a company. The input into the model has been checked and curated to contain minimal differences between protected groups (e.g., age, ethnicity or gender). However, in a real-life example of Simpson’s Paradox, it is possible to find bias in a success model for salespeople in Ohio, even if the same model would not have bias on a global population. The presenter will discuss how this is addressed at Pymetrics.

Julie Dawson, Director of Regulatory & Policy, Yoti: Yoti, a digital identity platform and a BCorps, provides secure AI identity and age estimation in a way that protects user's privacy and is transparent about accuracy across gender, age and skin tone. The presenter will share how the platform has applied ML research to age estimation and steps being taken to address bias and be open about the levels of bias to relying parties and regulators.

Luke Stark, Postdoctoral Researcher, Microsoft (with Michael A. Madaio, Jennifer Wortman Vaughan, & Hanna Wallach): Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI. Many organizations have published principles and even checklists intended to guide the ethical development and deployment of AI systems; however, unless checklists are grounded in practitioners' needs, they may be misused. To understand the role of checklists in AI ethics, Microsoft conducted an iterative design process with practitioners to co-design an AI fairness checklist, and identifying desiderata and concerns for AI checklists in general. It was found that AI checklists could provide organizational infrastructure for formalizing ad-hoc processes and empowering individual advocates, and highlight future research directions in this space.

Sarah Aerni, Director of Data Science and Engineering, Salesforce: Salesforce Einstein democratizes AI by putting it in the hands of business experts with little experience or understanding of machine learning. The presenter will cover how Salesforce provides an appropriate level of friction in the setup experience, automates the modeling process, builds guardrails to protect users from common mistakes, and provides scorecards to offer broadly-consumable explainability to help users in their next steps.