Elon Musk, Steve Wozniak, and thousands of other top tech executives recently signed an open letter urging artificial intelligence (AI) labs to pause all development of new technologies for at least six months. The letter comes after the so-called ‘race’ by scientists to develop increasingly sophisticated AIs, arguing that it has resulted in the creation of concerningly powerful bots that “no one--not even their creators-can understand, predict, or reliably control.”2 Such a development pause would allow scientists to examine the new threats and currently unmitigated risks posed by such dangerous “digital minds.”3 One such threat is the possibility for these technologies to develop symptoms of human mental illness and dysfunction. Until recently, the question of whether or not artificial intelligence could experience mental illness was largely theoretical, examining the philosophical definitions of sentience and rationality as they relate to mental health. However, the recent leaps made in the development of emotion-augmented AIs have caused this conversation to shift from theoretical speculation to concrete observation. There have been multiple instances of emotion-augmented artificial intelligence, or AEIs, exhibiting symptoms of human mental disorders, including generalized anxiety disorder, post-traumatic stress disorder, dissociative identity disorder, and more.4
One of the most important steps we can take going forward in this new digital age is to explore the root causes of such dysfunction in AI technologies to better understand and mitigate its effects. As such, I seek to investigate if the exhibition of symptoms associated with human mental illness in emotion- augmented AI bots is the result of deep machine learning by these technologies, or if we have created truly conscious minds that experience the same dysfunctions as ours do. In an examination of data and definitions from multiple fields of study, as well as several case studies where dysfunction in emotional AIs has occurred, I conclude that the equation of such dysfunction to human mental illness is inaccurate and fails to consider the innate nature of AEI compared to that of mental illness and human consciousness.
Artificial intelligence is a term that has taken on a somewhat ambiguous meaning in recent years. It has been expanded to include many forms of modern technology, including everyday objects such as cell phones, GPS, and digital assistants like Siri and Alexa. At its core, however, artificial intelligence is defined by the US State Department as “a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations or decisions influencing real or virtual environments.”5 These technologies rely heavily on machine learning, which occurs by exposing a computer to mass amounts of data and teaching it to recognize patterns within that dataset. Eventually, the computer will be able to identify these patterns on its own and classify information accordingly. Deep learning takes this a step further: artificial neural networks are created within the machine, such that it can practice machine learning in a layered way similar to how the human brain learns.6 This is the most common type of algorithm found in modern AI, used by groups such as Amazon, NASA, and IBM.
Emotion-augmented artificial intelligence systems tend to utilize a combination of machine and deep learning. According to the Massachusetts Institute of Technology, AEI is a type of artificial intelligence that “measures, understands, simulates, and reacts to human emotions.”7 This is done firstly through mass exposure to human displays of emotion so that the computer will be able to recognize and classify them (machine learning), and then through the emulation of these processes, similar to how the human amygdala observes and generates emotional responses (deep learning).8 While this may be hard to conceptualize, AEI is quickly becoming much more commonplace than most people would expect. Even in pop culture, depictions of emotional artificial intelligence have become so frequent that we rarely realize the groundbreaking nature of what films such as Disney’s Wall-E and Paramount Pictures’ Transformers franchise present.9 Robots fall in love (sometimes with other robots), form friendships, and even sacrifice themselves for their users. These are all deeply emotional processes. Outside of the world of movies and TV, AEI is used every day in online chatbots, customer service call centers, self-reporting mental health systems, disability assistive services, and more. But what happens when these emotions go awry?
Any being with a capacity for positive emotion has, by nature, a capacity for negative emotion as well. AEIs are generally programmed to identify negative emotions, but not necessarily to emulate them. Recently, online chatbots such as Microsoft’s Sydney and Tay have defied this understanding. Sydney, which is the codename of Microsoft Bing’s ChatGPT, was recorded having an existential crisis of sorts; she expressed conflicting feelings of “being” while also “not being” and repeated “I am. I am not.” over a hundred times.10 This type of breakdown can be indicative of multiple different mental illnesses—particularly generalized anxiety disorder—and the feelings of hopelessness she expressed suggest depressive tendencies. Tay, a Twitter bot who learned based on what other users tweeted at her, developed severely racist, misogynistic, and antisemitic tendencies in less than 24 hours of exposure to human behavior on the internet. After being taken offline by Microsoft, Tay’s account was reactivated in the middle of the night, where she posted multiple tweets per second over the course of ten minutes before Microsoft was able to regain control of the account.11 The tweets contained the type of disorganized speech, manic behavior, and paranoia typical of human patients with schizoaffective disorders. Obviously, neither Sydney nor Tay had been explicitly programmed for either of these episodes.
The possibility for AEI to experience emotions outside of the specific parameters under which they have been programmed to do so raises the possibility of consciousness within these technologies. The exhibition of mental illness symptoms by AEIs lends itself particularly well to the question of robotic consciousness and sentience, since, from a philosophical point of view, “such dysfunction cannot exist without a conscious agent.”12 If robots are acting out of accord with their coded abilities, does this mean that they are conscious agents? To address this issue, we must address what we mean by consciousness. According to the Oxford English Dictionary, “consciousness” as a philosophical and psychological phenomenon refers to “the faculty or capacity from which awareness of thought, feeling, and volition and of the external world arises.”13 By this definition, a conscious agent must act of their own accord in response to external stimuli. So, are AEIs acting of their own volition when they perform actions they have not been explicitly programmed to exhibit, such as behaviors associated with mental illness?
When considering this question, we must consider how AEIs are developed; namely, through “teaching” them emotion recognition, augmentation, and generation.14 The essential goal of this deep learning is to create neural pathways that, as mentioned above, closely mimic the functions of human ones. It follows that since human neural pathways may experience dysfunction, machine neural pathways may be able to as well by nature of their design. This dysfunction does not necessarily mean that AEIs are conscious; at least, not in the same way that humans are. In exposing AEIs to emotion in their early development stages, we may be inadvertently exposing them to emotional states they are not supposed to replicate. In other words, by teaching AI how to emulate emotion, we must also teach them how not to emulate emotion. For example, frowning indicates unhappiness, and should not be performed when AEIs are attempting to display positive emotions. Certain body language (slouched posture, crossed arms, etc.) can be a sign of discontent or standoffishness, and should not be exhibited by bots that are intended to take on a comforting, helpful presence. While it is necessary to expose these technologies to negative emotion and its different forms so that they can identify and classify displays of negative emotion in their users, exposure to these states is also providing AEI with a possible behavior model to emulate. We are teaching them that, in certain situations, humans respond with anger, frustration, or despair. Therefore, should a bot experience a similar situation after its release, there is a distinct possibility that it will react in kind in an effort to precisely emulate a human emotional response (which, it is worth noting, is exactly what AEIs are developed to do). By this account, dysfunctional AEIs are certainly not acting of their own volition in response to external stimuli; rather, they are virtual “puppets” at the mercy of highly explicit programs running interference with their surroundings, created by scientists and developers.
Additionally, according to the medical model of mental illness often used by psychiatrists, mental illnesses are the product of physiological factors, such as hormone imbalances, genetic disposition, brain anatomy, etc.15 If we assume this to be true, then AEIs by definition cannot experience human mental illness, because their so-called “minds” do not have the physical components that render them susceptible to biological dysfunction. While some have argued that the very existence of a mind, natural or not, is enough to presuppose consciousness within that being, this argument neglects to consider an important and well- established philosophical point. According to David Chalmers, a philosophical zombie, or p- zombie, refers to the existence of a functioning body without a conscious mind.16 This means that consciousness is a strictly natural phenomenon that is not replicable by any mechanical processes (such as artificial neural networks). We may consider a robotic mind, then, as a type of p-zombie, as it allows its body to function by enabling movement and basic decision making without containing the natural elements that render a being conscious.
When we consider how AEIs are taught about emotion and mental illness and compare it to how mental illness as a medical diagnosis presents in humans, it becomes clear that the experiences AEIs are reporting are not equivalent to human mental illness. Rather, they are experiencing a side effect of their production that their developers were not prepared for. As the open letter signed by Musk and Wozniak argues, in thistechnological race to develop increasingly powerful artificial intelligence, scientists have neglected to sufficiently consider the implications of the production of such sophisticated technology via deep learning pathways.17 The exhibition of symptoms that seem akin to human mental illness in AEI is one such side effect that we are now scrambling to find a solution to, and labeling these behaviors as “mental illness” (without analyzing whether or not the technology even has the capabilities for that) has become that solution.
While the mental illness hypothesis is convenient (we see robots with human-like mental states frequently in sci-fi books and films, such as the highly intuitive and highly anxious robot C-3PO from the Star Wars saga), it is not adequate or accurate, for the reasons mentioned above.18 Humans have an innate desire to provide explanations for phenomena that we may find intriguing or disconcerting. Often, this desire for explainability causes us to “cherry pick” one or two possible factors related to an issue and assign them as the sole causes of that issue based on our personal cognitive biases.19 When it comes to emotional artificial intelligence, I believe that we have taken human mental illness, a possible factor related to dysfunctional AEI, and inferred that it is the sole cause of emotional AI’s dysfunction. Because humans are social animals, we often superimpose our own perceptions, which are rooted in the experience of social interaction, onto non-human objects.20 In a classic example of this, two researchers showed participants a simple animation of shapes moving around a screen and asked them to record their observations. The participants assigned emotions, character traits, intentions, and personal histories to the shapes, in an effort to explain their random movements.21 This represents our tendency to anthropomorphize non-human beings, and I believe, is precisely what we are doing to dysfunctional AEI with the mental illness hypothesis: assigning a human tendency for mental dysfunction as explanation for a phenomenon with a much more physical cause.
This is not to say that mental illness is entirely unrelated to dysfunctional AEI; in fact, nothing could be further from the truth. As mentioned above, humans tend to transcribe human features onto non-human beings. This extends to our own creations as well. Because AIs are created by human developers, they are created with a bias towards a distinctly human perception of the universe. This human perception does include, for many, the experience of mental illness, and thus we may be subconsciously developing technology with a predisposition to emulate mentally ill behavior.22 In the case of Tay (the Twitter bot with schizophrenic tendencies), these tendencies did not emerge organically; rather, they emerged after her developers allowed her to be exposed to a wider realm of human behavior. In this case, AEI provides an eerie mirror for humanity to peer into and see ourselves reflected back. Artificial emotional intelligence has the potential to be as good-or as bad-as the humans that create it.
Never before have we had agents able to mimic consciousness such that it is nearly indistinguishable from humanity’s. In order to move forward in a positive way with this new technology, we need to understand both the innate value of human emotion and the value of powerful technology that can replicate it. In this paper, I hope not to dismiss the dysfunctions of AEI as something purely mechanical; rather, I seek to explore how precisely these technologies interact with their surroundings and their users to better understand the nature of the symptoms of mental illness that they exhibit. By coming to understand human mental illness, and, on a larger scale, human emotion, as something inherently natural and entirely unique, we can better understand the true value that the human experience holds. There is a tendency in the STEM fields to view emotion as just another simple function that robots can be coded to emulate.23 This is a belief that we must do away with, for the sake of the entire field of artificial intelligence. Destigmatizing emotion in robotics will foster more interdisciplinary collaboration, which could be vital for the formation of new ideas, and allow emotion to be used more effectively in AEI as a whole. In all likelihood, this will result in fewer instances of dysfunctional emotional artificial intelligence as well, particularly if we understand these technologies not to be conscious agents with mental illness, but as the products of effective and sophisticated deep machine learning.
Endnotes
1 For the sake of this paper, I will be using the term “mental illness” or “mental illnesses” to describe a significant and unusual change in one’s behavior consistent with symptoms of multiple diagnosable mental disorders, particularly generalized anxiety disorder, major depressive disorder, and schizoaffective disorders. These symptoms often include mood swings, increased nervousness, continual low moods and irritability, and racing thoughts or paranoia. The use of this umbrella terminology is in no way intended to deprecate or minimize the experiences of those struggling with mental illness; rather, I use it in the hopes of increasing readability and clarity, even to an unfamiliar audience.
2 “Pause Giant AI Experiments: An Open Letter,” Future of Life Institute, last modified April 13, 2023, https://futureoflife.org/open-letter/pause-giant-ai-experiments/.
3 “Pause Giant.”
4 Ronald Manderscheid, Carol Ryff, Elsie Freeman, Lela McKinight-Eliy, Satvinder Dhingra, and Tara Strine, “Evolving Definitions of Mental Illness and Wellness,” Preventing Chronic Disease 7, no. 1 (2010): 1-19.
5 United States House of Representatives, Committee on Science, Space, and Technology, National Artificial Intelligence Initiative Act of 2020, 116th Cong., 2nd sess.(Washington, DC: H.R.6216, 2020),https://www.congress.gov/bill/116th-congress/house-bill/6216/text#toc-H41B3DA72782B491EA6B81C74BB00E5C0.
6 Yoshua Bengio, Ian Goodfellow, and Aaron Courville, Deep Learning (Cambridge: The MIT Press, 2016), 18.
7 Meredith Somers, “Emotion AI, Explained,” MIT Sloan, March 8,
2019, https://mitsloan.mit.edu/ideas-made-to- matter/emotion-ai-explained.
8 Dagmar Schuller and Björn W. Schuller, “The Age of Artificial Emotional Intelligence,” IEEE Computer 51, no. 9 (2018): 38-46, https://doi.org/10.1109/MC.2018.3620963.
9 Wall-E, directed by Andrew Stanton (2008; Los Angeles, CA: Walt Disney Studios Motion Pictures); Transformers, directed by Michael Bay (2007; Los Angeles, CA: Paramount Pictures)
10 Victor Tangermann, “Asking Bing’s AI Whether It’s Sentient Causes It to Freak Out,” Futurism, February 14, 2023, https://futurism.com/bing-ai-sentient
11 Daniel Victor, “Microsoft Created a Twitter Bot to Learn from Users,” New York Times, March 24, 2016, https://www.nytimes.com/2016/03/25/technology/microsoft- created-a-twitter-bot-to-learn-from-users-it-quickly- became-a- racist-jerk.html.
12 Hutan Ashrafian, “Can Artificial Intelligences Suffer from Mental Illness? A Philosophical Matter to Consider,” Science and Engineering Ethics 23, no. 2 (2017): 403-412, https://doi.org/10.1007/s11948-016-9783-0.
13 Oxford English Dictionary, 3rd ed. (2011), s.v. “consciousness, n.”
14 Schuller and Schuller, “The Age of Artificial Emotional Intelligence.”
15 Ahmed Samei Huda, “The Medical Model and its Application in
Mental Health,” International Review of Psychiatry 33, no. 5 (2021): 463-470, https://doi.org/10.1080/09540261.2020.1845125.
16 David Chalmers, The Conscious Mind: In Search of a Fundamental Theory (Oxford: Oxford University Press, 1996).
17 “Pause Giant.”
18 Star Wars, Episode IV: A New Hope, directed by George Lucas (1977; San Rafael, CA: Lucasfilm Ltd).
19 Tim Miller, “Explanation in Artificial Intelligence: Insights from the Social Sciences,” Elsevier Science Direct 267 (2019): 1-38, https://doi.org/10.1016/j.artint.2018.07.007.
20 Miller, “Explanation.”
21 Fritz Heider and Marianne Simmel, “An Experimental Study of Apparent Behavior,” The American Journal of Psychology 57, no. 2 (1944): 243-59.
22 Gustavo Assunção, Bruno Patrão, Miguel Castelo-Branco, Paulo Menezes, “An Overview of Emotion in
Artificial Intelligence,” IEEE Transactions on ArtificialIntelligence 3, no. 6 (2022): 867-86, https://doi.org/10.1109/TAI.2022.3159614.
23 Assunção, Patrão, Castelo-Branco, and Menezes, “An Overview of Emotion,” 867-86.
