- One reason for concentrating initially on "What" a system should do, before turning our attention to "How" it should do it,
- is that we want our system specifications to be free of Implementation Bias.
- It allows developers and their customer to focus discussion on the customer's requirements,
- without introducing irrelevant and distracting material about the workings of the computer system.
- When the implementation stage of development is reached it allows the developers more freedom
- because their implementation choices have not been pre-empted in the specification.
Categories of Bias in Computer System Design:
- Pre-existing Bias:
- Pre-existing bias has its roots in social institutions, practices, and attitudes.
- When computer systems embody biases that exist independently, and usually prior to the creation of the system, then the system exemplifies preexisting bias.
- Preexisting bias can enter a system either through the explicit and conscious efforts of individuals or institutions, or implicitly and unconsciously, even in spite of the best of intentions.
- Individual: Bias that originates from individuals who have significant input into the design of the system, such as the client commissioning the design or the system designer.
- Societal: Bias that originates from society at large, such as from organizations, institutions, or culture at large.
- Technical Bias:
- Computer Tools: Bias that originates from a limitation of the computer technology including hardware, software, and peripherals.
- De-contextualized Algorithms: Bias that originates from the use of an algorithm that fails to treat all groups fairly under all significant conditions.
- Random Number Generation: Bias that originates from imperfections in pseudorandom number generation or in the misuse of pseudorandom numbers.
- Formalization of Human Constructs: Bias that originates from attempts to make human constructs such as discourse, judgments,
- or intuitions amenable to computers: when we quantify the qualitative, discretize the continuous, or formalize the non-formal.
- Emergent Bias:
- Emergent bias arises in a context of use with real users.
- This bias typically emerges some time after a design is completed, as a result of changing societal knowledge, population, or cultural values.
- User interfaces are likely to be particularly prone to emergent bias because interfaces by design seek to reflect the capacities, character, and habits of prospective users.
- New Societal Knowledge: Bias that originates from the emergence of new knowledge in society that cannot be or is not incorporated into the system design.
- Mismatch between Users and System Design: Bias that originates when the population using the system differs on
- some significant dimension from the population assumed as users in the design.
- Different Expertise: Bias that originates when the system is used by a population with a different knowledge base from that assumed in the design.
- Different Values: Bias that originates when the system is used by a population with different values than those assumed in the design.