Tools

List of known frameworks, toolkits, checklists, and principles

Frameworks

• Ethical OS Framework by IFTF & Omidyar Foundation
  • "The Ethical Operating System can help makers of tech, product managers, engineers, and others get out in front of problems before they happen. It’s been designed to facilitate better product development, faster deployment, and more impactful innovation. All while striving to minimize technical and reputational risks. This toolkit can help inform your design process today and manage risks around existing technologies in the future. "
  • A few companies are beta testing right now
• Responsible AI in Consumer Enterprise by Integrate.ai
  • A framework to help organizations operationalize ethics, privacy, and security as they apply machine learning and artificial intelligence
• UK Data Ethics Framework by UK gov
  • Includes principles, guidance, & a workbook to record decisions made
• A Moral Framework for Understanding of Fair ML through Economic Models of Equality of Opportunity - paper by ETH Zurich
• An Ethical Framework for a Good AI Society:Opportunities, Risks, Principles, and Recommendations by AI4People
  • "We introduce the core opportunities and risks of AI for society; present a synthesis of five ethical principles that should undergird its development and adoption; and offer 20 concrete recommendations to assess, to develop, to incentivise, and to support good AI, which in some cases may be undertaken directly by national or supranational policy makers, while in others may be led by other stakeholders. If adopted, these recommendations would serve as a firm foundation for the establishment of a Good AI Society"
• Draft AI Ethics Guidelines For Trustworthy AI by European Commission’s High-Level Expert Group on Artificial Intelligence (AI HLEG)
  • Includes a framework for Trustworthy AI , principles and values, methods, and questions for assessing trustworthy AI.

Toolkits

• Ethics in Tech Toolkit for engineering and design practice by Santa Clara Univ. Markkula Center
  • "Each tool performs a different ethical function, and can be further customized for specific applications. Team/project leaders should reflect carefully on how each tool can best be used in their team or project settings. "
• An introduction to software engineering ethics -- free course
• Ethics & Algorithms Toolkit by Data Science SF
  • A risk management framework for governments (and other people too!)
  • Contains link to research paper they wrote with lots of references
• AI Fairness 360 by IBM
  • Open source with case studies, code, and anti-bias algorithms, tutorials, demo
  • White paper
• Design Ethically Toolkit by Kat Zhou (IBM)
  • A library of exercises and resources to integrate ethical design into your practice.
• Algorithmic Accountability Policy Toolkit - AI Now
  • "The following toolkit is intended to provide legal and policy advocates with a basic understanding of government use of algorithms including, a breakdown of key concepts and questions that may come up when engaging with this issue, an overview of existing research, and summaries of algorithmic systems currently used in government. This toolkit also includes resources for advocates interested in or currently engaged in work to uncover where algorithms are being used and to create transparency and accountability mechanisms."
• World Economic Forum’s AI Board Toolkit (still yet to be released)
  • "This project aims to create a toolkit for corporate boards to identify specific benefits of AI for their companies and concrete ways to design, develop, and deploy it responsibly."

Checklists

Deon: An ethics checklist for data scientists

• "deon is a command line tool that allows you to easily add an ethics checklist to your data science projects. We support creating a new, standalone checklist file or appending a checklist to an existing analysis in many common formats."

O’Reilly: Data Ethics Checklist

Ten Simple Rules for Responsible Big Data Research

• "This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication."

Principles

• The Ethics Center Ethical by Design: Principles for Good Technology
• Accenture Universal data ethics principles
• Google AI Principles and Responsible AI Practices
• Microsoft AI principles
• SAP AI Principles
• Partnership on AI Tenets
• FAT/ML Principles for Accountable Algorithms and a Social Impact Statement for Algorithms
• The Global Network Initiatives Principles
• Asilomar AI Principles
• Principles for Digital Development
• Royal Academy of Engineering Statement of Ethical Principles
• Partnership on AI Tenets
• Twitter Moments Principles

Oaths, Manifestoes, and Codes of conduct

• ACM Code of Ethics & Professional Conduct
• IEEE Code of Ethics
• The National Academies of Science, Engineering, & Medicine Data Science Oath
• UK Gov Chief Scientific Advisor, David King’s Universal Ethical Code for Scientists
• Ethical Design Manifesto
• Manifesto for Data Practices
• NeverAgain.Tech Pledge
• The Programmer’s Oath

Policy Papers and Resources

• Access Now/RightsCon The Toronto Declaration
• University of Montreal Montreal Declaration for Responsible AI
• European Group on Ethics in Science and New Technologies Statement on Artificial Intelligence, Robotics and ‘Autonomous’ Systems
• Intel AI policies white paper
• IEEE Ethically Aligned Design for AI
• IDEO Design, Ethics & AI
• Fast.ai resources
• New Zealand Human Rights Commission: Privacy, Data and Technology: Human Rights Challenges in the Digital Age
• Mark Latonero (Data & Society) Governing Artificial Intelligence: Upholding human rights and dignity
• Accenture papers:
  • Facilitating ethical decisions throughout the data supply chain
  • Informed consent and data in motion
  • The Ethics of Data Sharing: A guide to best practices and governance -- 100 day & 365 day plans to implement change
  • Digital Trust: Strengthening customer relationships through ethics and security -- 100 day & 365 day plans to implement change