Chapter 8: Computer Reliability

Computer Reliability in the film:

Computer Reliability is depicted in the film by Rev-9 Terminator by only seeing the mission and trying to complete it without considering other factors. The Terminator kills Dani's family, which includes her brother and father, and US Border Patrol agents to try to reach and kill its target, Dani Ramos. It also doesn't account for all the damage that it does. This includes the automobile assembly plant, the various vehicles that get destroyed, personal property, like the shed, crashing a drone, and hacking of all the data to locate one person. Though this means that the computer is focused on its task, meaning that it is very reliable, it also suggests that there are consequences in the real world that need to be accounted for. This makes the Rev-9 Terminator unreliable in the real (human) world. 

Computer Reliability can also be seen in the film Legion, an advanced AI system. Legion was created for cyberwarfare. However, over time, through possibly bad/biased data, faulty algorithms, and physical data that was fed, Legion data suggested that without humans, there could be peace, so it turned against humanity. As the outcome of the computer changed from helping with cyberwarfare to eradicating humans, the computer system became very unreliable.

Additionally, the T-1000 Terminator Carl became unreliable in the eyes of Skynet/Legion. He was built by Skynet with the purpose of contributing to Skynet's world takeover by eradicating Sarah Connor's son, John. However, after he finishes this task, he starts to change and, in the end, protects the humans that Skynet and Legion have been trying to eradicate, which is the opposite of what he was meant to do. By leaving Carl without another purpose, Skynet relied on the fact that he himself would not evolve and didn't consider what might happen if he succeeded.

Real-world Parallels:

Boeing 737 Max:

Because of how the Boeing 737 Max was built, the plane needed a stabilization system called MCAS, which used an Angle of Attack (AoA) sensor to prevent stalls automatically. However, the AoA sensors were notorious for being easily damaged, and Boeing relied only on a single sensor [1]. This caused many issues, including 2 plane crashes resulting in the death of 346 passengers.  It is theorized that the AoA fed the system erroneous data and forced the plane to continuously nose dive, with pilots unable to take control [2]. Boeing's over-reliance on this one sensor caused many deaths and could have been easily avoided had they had multiple sensors and fail safes.

Therac-25:

The Therac-25 was a medical machine that used lasers for cancer treatment. However, the machine had multiple malfunctions and would set the radiation levels far too high, resulting in burns and eventual deaths. Previous versions of the machine used mechanical safety locks that prevented overexposure, but this version implemented these safety constraints only in the form of software with an onboard computer.  As a result, because of the many bugs in the software, the machine would overexaggerate the patients [3]. This goes to show that there always needs to be fail safes because relying only on the software can result in catastrophe. 

References:

1. Curt Devine and Drew Griffin, "Boeing relied on single sensor for 737 Max that had been flagged 216 times to FAA" (CNN, May 01, 2019) https://www.cnn.com/2019/04/30/politics/boeing-sensor-737-max-faa/index.html (April 30th, 2024) 

2. Gregory Travis, "HOW THE BOEING 737 MAX DISASTER LOOKS TO A SOFTWARE DEVELOPER", (IEEE Spectrum, April 18, 2019) https://spectrum.ieee.org/how-the-boeing-737-max-disaster-looks-to-a-software-developer (April 30th, 2024)

3. Apgar, C., & Prentice, R. (Mcombs School of Business 2023, February 15). "Therac-25. Ethics Unwrapped." https://ethicsunwrapped.utexas.edu/case-study/therac-25#:~:text=AECL%20sold%20eleven%20Therac%2D25 (April 30th, 2024)