Online Proctoring using Proctorio and SPSS or R etc.

Online Proctoring at the VU

Administering exams with monitoring is well possible. The VU has the combination of TestVision and Proctorio at its disposal to organize this. See this page and this page with detailed information on the Canvas Course Online Education.

This page does not deal with proctoring using Zoom. For that info, see this page and then the Zoom Teacher guide.

Using SPSS, Excel, R, RStudio and other applications

It is possible to use programs like SPSS, Exel, R in combination with TestVision and Proctorio. Students can start an exam by logging in to TestVision, start Proctorio and then commence the exam. Parallel to this, students can run SPSS etc. to do calculations based on questions in the TestVision exam. 

Faculty Keyuser

Work closely together with your faculty keyuser. This is important because the faculty keyuser knows a lot about TestVision and Proctorio and has short lines with the central support staff. The keyuser helps you in making decisions how to set-up the exam and help you to communicate with your students.

Cautions

When administering this kind of exams, a lot of things need to be considered and mitigation of risks is very important.

Disconnections

Because a student has to run the webbrowser Chrome with Proctorio AND the additional application, computing resources become scarce. So, both the TestVision/Proctorio processes can suffer and the additional application. This can result more easy in disconnections in which TestVision and Proctorio think that the student has left the exam. The exam will disconnect and the student has to log back in.

"The attempt ended when the student disconnected from Proctorio for 2 minutes" end message usually indicates that this type of submission occurs when the student has an abrupt technical issue (loss of power or internet connection, computer freezing).

Freezing

Because of the same reasons as above, chances are higher that students experience sessions in which applications can freeze.

Wrong keyboard, language, country and numerical settings

In R/SPSS/etc.,  the syntax is in English, but in Excel the syntax can be English (COUNTIF), Dutch (NUMBER.ALS), Hungarian (DARABTELI), etc. There are also differences in comma/point and semicolon/comma. For example POWER(3.14,2) versus POWER(3.14;2). 

As we have students from all over the world, oblige students to only use the English and Dutch versions of the OS or their locale settings on their computer. Also, you have to take this into account for possible automatic recognition of numerical values that are entered as text string answers.

Problems with downloading a data file

Send the datafile 24 hours in advance via Canvas, with a call to download it and read it in R/SPSS/Matlab/etc. This prevents the problem that students who are less computer savvy from not finding the downloaded file. In practice, a number of students are unable to find a file on their computer when it has been downloaded. Sending it prior to the exams ensurens sufficient time for students to know where the file is when the exam starts.

Design the exam

How to provide the question and data

The first way to provide students the questions is through the question types available in TestVision. However, you can also give the questions by making an PDF with the questions that students can download from TestVision.

You can also put data in a file that students can download from TestVision to their own computer. Be aware that the data that you provide in this way will mean you lose control over that data. If you use the same dataset for your follow-up exams, that is then no longer possible.

Different computers an OSs

Students can have different computers with different operating systems (Windows, MacOS, ChromeOS maybe). Also, they can have different versions of the application. Also, students can have different screen resolutions. Make sure students can practice sufficiently and you take care of these differences in your instructions.

How students must respond to questions

You have to decide how students must respond to your questions. There are several possibilities

• Have students type an answer as a text string in TestVision via an open ended question which you score by hand. You can ask for program code for example. This is quite a fail safe method.

• Have students type all answers in a separate document (for example Word) which they then upload in TestVision upload question. This method is less fail safe a students have to open this additional application that can hence cause extra technical failure.

• You can ask your students to write the answers with a pen on a sheet of paper. Some domains simply need this, for example in domains involving writing maths or drawing diagrams. Students then have to scan their answers and then export it to a PDF or JPG file. These can then be uploaded into on upload question in TestVision. This approach can cause a lot of stress and technical failure. Make absolutely sure that students can practice this method. Also have a backup plan ready for students who or not able to upload. Make sure that they also can mail the documents or upload them via Canvas for example.

Scanning answers from a paper

Be aware that not all students have scanning equipment or know how to operate it and get PDFs out. So students must be able to practice.

Students mostly can scan with their mobile phone. We recommend the application ClearScan / ClearScanner as it seems the easiest to use and it directly combines multiple scan into one document.

Communication and Practicing

Communicate clearly with your students

Take your students along in your journey for them to take this kind of online proctored exams. Be fair. Point your students to the Online Proctoring website.

Provide clear instructions

Provide your students with clear instructions. First, you have to provide it upfront via Canvas. Second, you have to put these instructions in the Explanation/Introduction page of TestVision.

Design a Practice Exam

Six general practice exams are available for all students at the VU. These practice exams however do not deal with students running SPSS or other applications parallel to the exam. You must make a practice exam for your students that exactly mimics the way in which your students will do the exam.

Test Test Test Test ... and Test

An imperative part of your work must be about testing. Make sure to run through the whole process in detail and simulate/execute it as it would be done for real. Design your exam and test everything through and through! Involve Guinee pig students who do not know anything about your procedure. They will spot omissions you would not have thought of!

Real-time support

You have to inform the people of the central support staff that are available during the exam thoroughly. You have to provide clear instructions to this team so they know what the students have to do and how they could help them. Be aware that the support staff do not have knowledge about using SPSS etc. and are hence not able to support students with such problems.

Surveillance

You have to inform the people of the central proctor pool (VU surveillanten pool) how they must monitor your students.

The first thing to be aware of is that the proctoring software does not lock the computer of the student. So, students can access whatever resource that is available.

Be aware that the proctors can only generally monitor what students do on their computer with SPSS kinds of exams. The proctors cannot monitor every detail or action that students perform on their computer.

The main effect of online proctoring is THAT students are reviewed afterwards and that spotting fraude is possible if suspected as afterwards all details of the activities on the screen can be studied in detail.