From cognitive bias to algorithmic racism

In 2015, A black software developer embarrassed Google by tweeting that the company’s Photos service had labelled photos of him with a black friend as “gorillas.” * In 2010, Nikon cameras were criticized when image-recognition algorithms consistently asked Asian users if they were blinking.

Let’s take a step back.

Our brains have evolved over millions of years to the marvel of efficiency we have today. Just think about that our best artificial intelligences can measure up to humans only when considering specific task. One of the key to the success of human brain, and its limitation is the ability to perform hundreds of thousands operations automatically, this includes the metal shortcuts also known as heuristics. These shortcuts are useful for our brain to optimize some cognitive processes, but unfortunately, many times, they lead it to elaborate wrong choices. These little gimmicks are called cognitive biases. Furthermore, let's clarify immediately that we must not confuse bias and heuristics, in fact if the cognitive biases are the final error, the heuristics are the cause or if you want, the mental process that led us to that error of evaluation.

As we mentioned earlier, our brain wants to spend little energy and wants to make a decision as quickly as possible. Without going into too much detail let us take an example to better understand the concept of bias. We believe that, at least once in our life, we have all found ourselves to face the judgments when changing jobs. In this situation it may happen to be quickly judged by new colleagues, for example based on country of origin.

Surely very few of these quick judgments will be correct, or rather none is based on a thorough knowledge of you as a person. For humans quick judgments work in sync with other functions of the brain including instinctual reactions and are an essential part of our evolutionary success. In a situation where time is limited, the most resource efficient path will be the quick judgment. All in all, well formulated judgements and quick spontaneous decisions are tools available to our brain. Let us now move to computational biases.

From a mathematical point of view, the bias of a neural network, associated either with a single neuron or with a hidden layer, provides a linear variation of any exact value.

The bias translates into our network the concept of distortion or deviation of the desired output, a bit like what happens in our brain as regards the cognitive aspect: a high distortion means that the model is not "adapting" well to the training set, this will result in a very large training error; low distortion, on the other hand, means that the model has adapted well and the training error will be low. Ultimately a value carried by the bias must be as small as possible as it is telling us that the network is processing the incoming data well and therefore can provide an assessment close to reality. For more information of the basic elements of a neural network can be found in the book connected to the blog, under completion at time of writing.

So how comes that the Google or Nikon algorithms can make such a macroscopic error in judgment? As we have seen, algorithms can sure be biased, but that is something that as designers can adjust by trial and error, a bit like we can train ourselves to reduce the weight of bias in our own decision-making. Algorithmic bias can emerge due to many factors, including design choices, unbalanced datasets used for training and treating the code as a black box. Perhaps an algorithm (or a neural networks) can be trained correctly to classify cats and dogs, but to distinguish more complex patterns networks of algorithms (systems of systems) need to cooperate with often unexpected results. Last but not least, the programmers use their brain and biased mental processes when designing the code, which translated into a biased algorithm!

*Wired magazine, 2017

**Rose, Adam (22 January 2010). "Are Face-Detection Cameras Racist?". Time. Retrieved 18 November 2017.