Responsible AI in Practice Course

Overview

Welcome to Responsible AI in Practice! The Privacy and Responsible AI course educates participants on ethical considerations, regulatory frameworks, and best practices for developing and deploying AI systems with a focus on protecting user privacy and ensuring responsible use of AI technologies. This course is offered as part of Stanford University’s Institute for Computational and Mathematical Engineering (ICME) Summer Workshops Series.

How do we develop machine learning models and systems taking fairness, accuracy, explainability, robustness, and privacy into account? How do we operationalize models in production, and address their governance, management, and monitoring? Given the increasing role played by artificial intelligence (AI) applications in determining our day-to-day experiences and the rapid adoption of AI systems in high-stakes domains such as hiring, lending, and healthcare, it’s become imperative to build and deploy AI systems in a responsible manner. In this course, we will first motivate the need for adopting a "responsible AI by design" approach when developing AI / machine learning (ML) models and systems for different consumer and enterprise applications. We will present an overview of responsible AI and associated techniques and tools, with a focus on model explainability, fairness, and privacy in AI. Then, we will focus on the application of explainability, fairness assessment/unfairness mitigation, and privacy techniques in industry, wherein we present practical challenges/guidelines for using such techniques effectively and lessons learned from deploying models for several web-scale machine learning and data mining applications. We will emphasize that topics related to responsible AI are socio-technical, that is, they are topics at the intersection of society and technology. The underlying challenges cannot be addressed by technologists alone; we need to work together with all key stakeholders — such as customers of a technology, those impacted by a technology, and people with background in ethics and related disciplines — and take their inputs into account while designing these systems. We also motivate the need for adopting responsible AI principles when developing and deploying large language models (LLMs) and other generative AI models, and provide a roadmap for thinking about responsible AI for generative AI in practice. We provide a brief technical overview of text and image generation models, and highlight the key responsible AI desiderata associated with these models. We then describe the technical considerations and challenges associated with realizing the above desiderata in practice.

In this course, you’ll get a first hand feel for responsible AI industry case studies, and discover the challenges underlying responsible AI in practice! Join us for an interactive learning experience that includes engaging presentations, collaborative group exercises, polls & quizzes, and even a "homework" to probe deeper between the two sessions.

Course Material

Course Slides (Google Slides link; embedded above)

Zoom Video Recording

1. Session 1: Part 1 and Part 2

2. Session 2: Part 1 and Part 2

Course Outcomes

After this class, you will:

• Understand the need and challenges of ethical consideration in AI/ML development.

• Develop a working knowledge of responsible AI principles and techniques, with a focus on model explainability, fairness, and privacy in AI.

• Build a practical awareness of risk assessment, harm reduction approaches, and emerging trends and best practices.

Prerequisites

This course is aimed at attendees with a wide range of interests and backgrounds, including researchers interested in knowing about techniques and tools for model explainability, fairness, and privacy in AI as well as practitioners interested in implementing responsible AI models for web-scale machine learning and data mining applications. We will not assume any prerequisite knowledge, and present the intuition underlying various explainability, fairness, and privacy notions and techniques to ensure that the material is accessible to all attendees. To knowledgeably appreciate what can go wrong when developing AI/ML systems and how such problems are addressed, it’s helpful to have a conceptual understanding of machine learning and how it differs from computer software developed by explicit programming. Previous exposure to machine learning at the level of the companion “Introduction to Machine Learning” workshop in this series should suffice. Beginners to ML will do well to read the transcript from this link: The Ethical Algorithm, with Michael Kearns. Warning: it will take a careful reader at least 15, if not 30 minutes or more!

Requirements

To join the workshop, you’ll need a device with a recent web browser and two-way audio and video access to Zoom. This could be a laptop or desktop computer running any operating system, such as Windows, Mac, or Linux. Participative activities benefit from a larger screen, so joining via a smartphone or tablet may not provide the best learning experience.

Course Outline

The course will consist of three parts: (1) responsible AI foundations including motivation, definitions, models, algorithms, and tools for explainability, fairness, and privacy in AI/ML systems (about 2 hours), (2) case studies across different companies, spanning different application domains, along with practical challenges and opportunities (about 2 hours), and (3) trustworthy generative AI (about 2 hours).

Foundations

• Motivation from regulatory, business, and data science perspectives

• Fairness-aware ML techniques/tools

• Explainable AI techniques/tools

• Privacy-preserving ML techniques/tools

• Open source and commercial tools for AI explainability, fairness, privacy, and ML model monitoring (e.g., Amazon SageMaker Clarify and Debugger, Fiddler AI's AI Observability Platform, Google AI Explainability, Fairness, and What-If tools, IBM AI 360 Toolkits, LinkedIn Fairness Toolkit (LiFT), Microsoft Fairlearn and InterpretML)

Case Studies (including practical challenges and lessons learned during deployment in industry)

We will present case studies across different companies, spanning different application domains. We will emphasize that topics related to responsible AI are socio-technical, that is, they are topics at the intersection of society and technology. The underlying challenges cannot be addressed by technologists alone; we need to work together with all key stakeholders --- such as customers of a technology, those impacted by a technology, and people with background in ethics and related disciplines --- and take their inputs into account while designing these systems. Finally, we will identify open problems and research directions for the data mining/machine learning community.

Trustworthy Generative AI

We also motivate the need for adopting responsible AI principles when developing and deploying large language models (LLMs) and other generative AI models, and provide a roadmap for thinking about responsible AI for generative AI in practice. We provide a brief technical overview of text and image generation models, and highlight the key responsible AI desiderata associated with these models. We then describe the technical considerations and challenges associated with realizing the above desiderata in practice.