Example 1: Facebook-Cambridge Analytica Data Harvesting
In 2018 it was revealed that over 87 million Facebook users’ data had been harvested by Cambridge Analytica to use for political purposes. This was done through Facebook’s API which was created with the intended purpose that “developers could now see social connections between people, and see the connections people have based on their interests and likes.” Facebook users consented to this data collection through Facebook’s terms, however they were not informed that their friends’ data would also be collected. The data collected included likes, location, demographic info, social connections, and behavioral signals.
This API could be used by any company, but was most notably used by Cambridge Analytica, a company that provided data-driven services to political campaigns. The company gained access to the data through an app called “thisisyourdigitallife.” This allowed access to around 320,000 users who consented to the data collection, though their friends did not consent. Cambridge Analytica used this data for psychological profiling of users and to create targeted political ads which influenced the 2016 U.S. presidential election and the Brexit referendum.
Ethical Problems
Inadequate Consent
Users consented to the data collection terms, but consent was not informed and their friends did not consent to their data being collected and used by the app.
Manipulation of Democratic Processes
The software influenced voter perceptions, political behavior, and public opinions.
Platform Design
Facebook’s API design allowed for broad 3rd-party access to data and minimal auditing of how developers use data.
Engineer vs. Product Responsibility
The software engineers likely were given instructions on how to execute the creation of the API and were unaware of potential political misuse. However, this suggests a lack of ethical consideration and a focus on growth rather than safety.
The software allowed for 3rd-party access to data which could be used in ways that users could not knowingly consent to. The software also did not have a system for protecting users’ privacy by limiting what user data could be used for.
Example 2: Uber Greyball
From 2014-2017, Uber used an internal software tool named Greyball. Uber stated the purpose of the software was to identify fake/fraudulent users and protect its drivers from harassment and threats. However, investigations revealed that the software was actually used to identify government officials and regulators in cities where Uber was banned. In order to do this Uber analyzed credit card metadata, geolocation patterns such as frequent location near government buildings, social media profiles, and repeated ride requests near restricted areas.
Once these accounts were identified, they were effectively “greyballed.” Greyballed accounts would be shown a fake version of the app with the ultimate purpose of preventing them from successfully booking a ride. Uber drivers found in restricted areas would be ticketed. Uber’s legal team argued that the software’s purpose of preventing drivers from harm included the harm of these tickets.
Ethical Problems
Intentional Deception
Greyballed accounts would be shown a fake version of the app including phantom cars which misrepresented ride availability.
Law Enforcement Evasion
The software was used with the explicit intention of evading local regulations and targeted government officials.
Illegal Competitive Advantage
Through the illegal operation of Ubers in restricted areas, taxis and competitors following regulations had an unfair disadvantage.
Engineer vs. Product Responsibility
Software engineers may have believed that the feature was intended to prevent fake/fraudulent accounts and not to target government officials. However, they still created a fake version of the app that would purposefully mislead users about ride availability and never provide the service the app is intended to give users.
The product itself is unethical as it was created to evade law enforcement and gave Uber an unfair advantage over other rideshare companies and taxi drivers. It was only implemented in cities with regulations, unfairly affecting customers and other drivers in these cities.
Example 3: Theranos
Theranos was a biotech startup founded in 2003 that claimed to be able to run hundreds of blood tests from a single finger prick. Theranos advertised this to be done by their original machines called Edison. It was a way for patients to receive blood test results in a cheaper, faster, and more accurate way than traditional testing. Medical reports included results about medical diagnoses, cancer risks, blood clot detection, and hormone levels.
However, the Edison machines were not able to accurately perform many of the promised tests and third-party machines were used without patients’ knowledge. Data was reformatted to appear as coming from the Edison machines rather than the third-party machines. Even with the use of more powerful third-party machines, the test results were inaccurate. Reports were still generated with the inaccurate results and delivered to patients.
Ethical Problems
Direct Medical Harm to Patients
False results were provided to patients while knowing that tests could be inaccurate. This led to panic, delayed treatment, and physical harm from unnecessary treatment due to inaccurate diagnoses.
Misrepresentation of Capability
Software was used to present unreliable results as clinically validated, mask hardware limitations, and falsify capabilities to investors and regulators. Theranos partnered with Walgreens and had a valuation of around $9 billion at its peak.
Engineer vs. Product Responsibility
Software engineers working on medical software have a responsibility to make sure that the data they are providing is accurate. In addition, engineers manipulated data to appear as if it was coming from the Edison machines rather than the third-party machines. Engineers also failed to report errors and falsified test accuracy, providing reports to patients.
The Edison machines were not able to run many of the tests and were not accurate. The machines were never able to be refined to successfully and accurately run the tests. They were not able to pass tests required for medical devices without the accuracy reports being falsified.