Who is Watching the Watcher?

Due to concerns about the rapid spread of the COVID-19 virus, “higher education institutions worldwide have moved their teaching online, as face-to-face teaching was banned” (Lee & Fanguy, 2022, p. 477).  One of the pedagogical concerns that accompanied this move online was how to effectively engage with assessment.  Formative and summative assessment, according to Meccawy, Meccawy, and Alsobhi (2021) is an integral part of leaning. Formative evaluation can drive, motivate, and enable learning to take place through the provision of feedback, while summative assessment can be used to measure student achievement (p. 3). Formal written examinations are a common type of summative assessment. Prior to the pandemic, many exams were closely supervised in the classroom, with strict protocols in place to discourage cheating. With the shift to online learning, avoiding cheating and academic dishonesty was a primary concern (p. 4).  While many educators were familiar with online platforms for formative evaluation, according to a study by Meccawy, Meccawy, and Alsobhi (2021), the “overwhelming majority (77.4%) had never used it for summative assessment” (p. 16-17). A common solution that was adopted was the use of online proctoring.

Online proctoring software “contains AI and machine learning components that analyze exam recordings to identify suspicious examinee behaviors or suspicious items in their immediate environment” (Coghlan, S., Miller, T. & Paterson, 2021, p. 1583). Students are required to have video cameras on, verify their identity, and show a 360-degree view of their surroundings.  In some instances, the camera feed is monitored by an instructor, and in others it is monitored by AI equipped software. The software identifies what it deems to be suspicious behaviour, (a student leaving the room, looking down or speaking aloud) and flags these instances.  Some AI programs can also detect cellphones or other devices that are present.  The instructor is notified of any flags and can review the recordings to inform their decision about whether the actions observed constitutes cheating. The recording can be used as evidence if the cheating is reported.  While concerns have been raised about the ethics of such surveillance practices, Lee and Fanguy (2022) explain that “in the ongoing Covid-19 pandemic, universities have not had better alternatives than using online proctoring products” (p. 478). Thus, many educators used systems such as Respondus, Monitor, or ProctorU to facilitate their exams.

According to Flaherty (2020), one Educause poll, indicated that “54 percent of institutions were using online or remote proctoring services, while another 23 percent were considering or planning to use them”.  Further, Flaherty explains that “twenty-six percent of institutions said they were using products that didn’t meet their accessibility standards.” Although OP software “first emerged in 2008” (Coghlan, Miller and Paterson, 2021), the growth of OP during the pandemic was unprecedented. Kharbat and Daabes (2021) outline how one “online proctoring company switched from having 100 customers per year to having 120 customers per day (p. 6590).  Although the growth of OP was widespread, the response was not monolithic nor entirely positive.

The implementation of OP is part of what Adedoyin and Soykan (2020) have called the ‘crisis-response migration’ to online learning.  They argued that despite the “undeniable fact … that online education has regularly been viewed from the perspective of good-to-have alternative but not a serious-mission model to guarantee steadiness of instructional activities” (p. 3), the global health crisis and the closure of schools made the shift to online unavoidable.  Further, lack of warning meant that there was an “absence of proper planning, design and development of online instructional programs due to the pandemic” (p. 8). Manokore and Kuntz (2020) concur, stating that “emergency virtual remote teaching should not be confused or conflated with traditional online learning” (p. 2).  Instead of having time to pre-plan, “Learning institutions, educators, and students were figuring things out in real time while also dealing with the stressors associated with a pandemic and waves of lockdown measures” (p. 2).  It is within this context that many decisions about what technologies to employ took place. 

Educational Ethics

In their comparison of emergency remote learning (ERL) and online learning, Manokore and Kuntz (2020) argue that in terms of delivery, ERL attempted to “mimic or replicate collaborative (classroom) teaching and learning” (p. 3). Therefore, to understand the shift to online proctoring, it is important to understand some of the educational values and paradigms that informed in-person learning.  Lee and Fanguy (2022), argue that “how universities managed and engaged with online educational provisions during the COVID-19 pandemic has effectively demonstrated the dominant educational paradigm controlling and regulating institutional practice before the pandemic” (p. 482).  Coghlan, Miller, and Paterson (2021) further argue that the values of “academic integrity, liberty, and trust – also have relevance to educational practice and educational philosophy” (1587). This is evidenced by Kharbat and Daabes (2021) who describe how the Ministry of Education (of the United Arab Emirates) decreed that “appropriate remote assessment tools should be put in place to preserve academic integrity and maintain educational standards” (p. 6591). Online proctoring tools were part of the arsenal adopted to maintain this integrity. 

Academic integrity involves “commitment from students, faculty, and staff to demonstrate honest, moral behavior in their academic lives” (International Center for Academic Integrity, 2021, as cited in Coghlan, Miller, and Paterson, 2021). Coghlan, Miller, and Paterson (2021) argue that although “educational institutions are partly motivated by reputational concerns, most nonetheless regard academic integrity as a vital intrinsic value” (p. 1588).  They identify four ethical reasons or rationales for the need for academic integrity.  

     First, Coghlan, Miller, and Paterson (2021), argue that the “value and viability of courses and universities depend on their academic integrity and educational rigor” (p. 1590). Thus, the proctoring of exams is necessary to maintain the reputation of the institution.  Second, cheating is seen as unfair to honest students. This ties into neoliberalism which Lee and Fanguy (2022) argue has an “exclusive emphasis on individuals’ accountability.”  Due to this emphasis, “the complex notion of educational ‘fairness’ became an individual student’s moral responsibility” (p. 484).  Exam proctoring, therefore, was understood to protect honest students from losing out to those who cheat. Cheating undermines another assumption of education, that it is built on a meritocratic system.  

According to Autin, Batruch, and Butera (2015), educational institutions have two main functions, education and selection (p. 2).  In addition to teaching skills and knowledge, education is part of a selection process that ranks and orders students according to their merit.  “The function of selection relies on a meritocratic ideal, whereby individuals are guided toward the position that corresponds to their dispositions” (p. 4). Normative assessment, of which exams are a part, serves the purpose of reducing “performance to a single indicator that is easily interpretable, which facilitates ranking and social comparison” (p. 3).  Thus, assessment is seen to serve the objective function of ranking that is necessary for meritocracy.  Cheating, thus, is seen to undermine meritocracy.

Morality forms the basis for the third and fourth rationales of academic integrity.  The third rationale is that “knowledge that others are cheating can create for honest students an invidious moral choice between self-interest (e.g., where class rankings matter) and personal integrity” (Coghlan, Miller & Paterson, 2021, p. 1590). Therefore, discouraging, and policing cheating is necessary to remove the temptation for ‘honest’ students.  This is related to the last rationale, that universities “bind themselves to providing students with (in some sense) a moral education alongside an intellectual education” (Coghlan, Miller & Paterson, 2021, p.1590).  The focus on academic integrity, therefore, means that “the failure to invigilate where necessary to prevent cheating above a certain level can, amongst other things, convey the impression that academic honesty is unimportant, thereby negatively affecting the institutional culture” (1590).  Further, as a study by Dendir and Maxwell (2020) suggests, students in online courses “tend to view proctoring as a sign that university considers cheating a serious issue, and this signal may cause students to alter their test-taking behaviours” (as cited in Lee and Fanguy, 2022, p. 478). It is within this context of the need to preserve academic integrity that OP technologies were considered and adopted. 

Liberty and trust are also cornerstones of educational philosophy, according to Coghlan, Miller and Paterson (2021).  Despite that OP technologies were widely adopted during the pandemic, some students “and university staff evidently feel that OP platforms could damage a university’s visible commitment to liberty and to earning trust” (1599).  Given that liberty and trust are concerned with “potential wider effects on freedoms, use of digital technologies, and society’s trust in AI, universities, etc.” (p. 1589), debates about its use have resulted from this adoption. While the use of OP is, at least in part, about the distrust of students, the adoption of OP required trust when it came to AI.  To understand this trust, it is important to understand the principles that inform AI ethics. 

The Ethics of Artificial Intelligence

In addition to the educational principles of academic freedom, liberty, and trust, Coghlan, Miller, and Paterson (2021) identified “fairness, non-maleficence, transparency, privacy, accountability, and respect for autonomy” (p. 1586) as principles that inform AI ethics.  It is to these principles that I will now turn, as beliefs about, and trust in AI technology, also contributed to the adoption of OP technologies. 

The idea that AI is free from bias, and is therefore fair, was promoted by companies such as Proctorio.  They argued that AI could “mitigate human bias and error … and surpass the human ability to accurately detect cheating” (as cited in Coghlan, Miller, and Paterson, 2021, p. 1586).  Thus, online proctoring was framed as not only a great alternative to human proctoring, but also, superior to human proctoring in terms of bias and efficiency.  Alternatives such as monitoring exams through Zoom were seen as less rigorous, as having an instructor monitor all students at once, especially in large classes, would have decreased the chances of catching cheaters.  Additionally, online proctoring was seen as fairer to students who, due to the pandemic, could not travel to exams or who were at greater health risks due to COVID-19.

Coghlan, Miller and Paterson (2021) define non-maleficence as the “effective and safe application of the technology which does not cause harm to the subject” (p. 1589).  Online proctoring was sometimes rationalized as being no different than being watched by an instructor in face-to-face exams. Morrison and Heilweil (2020) explain that advocates of “online proctoring say it’s simply recreating remotely what exam-takers would experience in person”. Additionally, they quote one instructor who minimized privacy concerns by saying that although some of her students weren’t happy about using the software, “she doesn’t think it’s worse than some of the data-collecting tools her students voluntarily use, like mobile phones and smart devices.” The idea that our data is always already being collected became a precedent for the data-mining of AI in online proctoring.  

The principle of transparency is also relevant here. According to Coghlan, Miller, and Paterson (2021), transparency refers “to the degree to which the determinations or predictions of AI systems are revealed to relevant parties in ways that those parties prefer and can understand (p. 1588).  The idea that educators and students can be made to understand how AI works in online proctoring is vital for those choosing to use the software.  However, transparency tends to be focused on what the software does, rather than how it does it.  Being transparent meant providing students with an overview of testing expectations, what behaviours would cause them to be flagged and the consequences of being flagged.  An important, and sometimes overlooked component of transparency is privacy.

According to Patael, Shamir, Soffer, Livne,  Fogel-Grinvald, and  Kishon-Rabin (2022), “Among the major challenges of remote proctoring are ensuring student identity, guaranteeing the integrity of the examinations, safeguarding students' privacy, managing students' anxiety” (p. 2). Although privacy issues were concerning for some students, a point I return to later, for faculty, “trust in the privacy of these eLearning systems was found to be the most decisive factor when considering the adoption of e-proctoring technology” (Kharbat & Daabes, 2020, p. 6592).  That some online proctoring software was built into the LMS systems that faculty were already using made that trust much easier. Any expected loss of privacy was rationalized as an acceptable limitation to student autonomy.

According to Coghlan, Miller and Paterson (2021), respect for autonomy “represents a broad commitment to allowing each individual to determine their own personal values and make their own choices, within the general framework of acceptable conduct determined by the society in which they live” (p.1588).  Given that educational culture is based on a commitment to academic integrity, acceptable conduct (non-cheating) was the framework that determined the limits of student autonomy.  Since allowing students to take an exam unsupervised could potentially violate this value, limiting student choices around whether or not be recorded was not seen as a serious violation of the ethic of autonomy.

Despite that it limited students’ autonomy, online proctoring was seen to hold students accountable for maintaining academic integrity.  Online proctoring software, as argued earlier, was seen as the best and most accurate way to ensure student accountability.  Lee and Fanguy (2021) found that student motives for cheating included incentive, opportunity, and rationalization (p. 478). While incentive and rationalization for cheating was the same in face-to-face and online exams, the opportunity for cheating in an online exam was viewed as increased “since many dishonest behaviours and actions may be harder for a proctor to detect within a teleconferencing environment” (p. 479). Further, accountability was not only about catching cheaters, but also about teaching students to police their own behaviours.  Online proctoring as a form of surveillance was understood to be more effective than face-to-face proctoring, as in addition to students being aware that they were being watched in the moment, they also were aware that their behaviours were recorded for review and to be used as evidence against those who were flagged.  Although Lee and Fanguy (2021) found in their study that surveillance in online proctoring was “fully justified and even welcomed by both parties involved in online exams” (p. 485), online proctoring has not been without its critics.

During the pandemic, emergency measures were necessary.  While many faculty felt that OP was their only viable choice if they were to preserve academic integrity, as we move forward, it is imperative that we take the time to consider the consequences of adopting this software.  We have to consider whether the technologies we use, including online proctoring, meet the ethical considerations of “fairness, non-maleficence, transparency, privacy, accountability, and respect for autonomy” (Coghlan, Miller and Paterson, 2021, p. 1586).  We also need to think seriously about what our roles as educators are when it comes to academic integrity.  As Stewart (2020) contends, “Academic integrity matters.  But integrity works both ways.” Exposing our students to ineffective forms of assessment and to the potential harm of these new technologies calls into question whether we are maintaining that integrity.