At the very start of the first lockdown, it became apparent that online exams have one big drawback: the lack of supervision. At first, a number of exams were postponed until the next period, but Maastricht University (UM) needed to prevent a build-up of study delays. Especially SBE, FHML and FSE department of Knowledge Engineering needed an online proctoring solution that would allow to monitor students while taking their exams. What has proctoring brought us, and how can we use it going forward?

​Finding a solution

In spring of 2020, the School of Business and Economics and the faculty of Health, Medicine and Life Sciences experimented with different setups using video conferencing also dubbed ‘proctoring light’. It soon became apparent that this method was too complex to be a viable solution: UM needed dedicated proctoring software.

As the result of a tender UM contracted Proctorio in September 2020. Proctorio is a browser extension for Chrome that is integrated with TestVision: as soon as a student starts a proctored exam, the plugin will record the student, their audio and screen during the entire exam, and it will cut off the connection to TestVision if the student tries to navigate away from the exam webpage. Afterwards a group of trained invigilators will review the recordings and report any irregularities to the Board of Examiners.

Proctorio in practice

Even with the integration that Proctorio offers, proctoring is not an easy solution. The privacy issues involved had to be evaluated carefully, and UM has taken measures to ensure the integrity of the recordings. The settings in Proctorio have been configured with care to only use the lockdown and recording features that are necessary.

The integration of Proctorio in TestVision means that setting up an exam is easy, but reviewing the recordings takes a substantial amount of time. The vast majority of irregularities are about students who did not follow the procedures correctly. For a proctored exam the students need to carry out an additional series of steps, all by themselves and with no-one to guide or correct them if they make a mistake. Even with clear step-by-step instructions it is essential that all students take a ‘practice exam’ to test their computer setup and practice the procedure.

Proctored exams rely heavily on technology, where the most important bottleneck is students’ hardware and bandwidth: up to 10% of test-takers lose the connection to their exam at some point. Even though the students can reconnect and continue their exam, exams are stressful enough as it is and it is frustrating to have to resolve technical issues at such a time. Non-proctored exams tend to run more smoothly, as these exams in general do not use as much bandwidth.

Furthermore, the software itself also has shortcomings. Immediately during the first exam week UM experienced a major incident, where hundreds of students had trouble opening their exams because the Proctorio servers could not handle the amount of traffic. This was followed by an even worse incident in December, when the new Proctorio server settings caused all exams in TestVision – also at other institutes – to become unresponsive. It seemed that with its explosive expansion, Proctorio had bitten off more than it could chew. The situation has since been remedied, but these incidents have understandably eroded trust in proctoring for students and staff alike.

Going forward: use with care

A couple of months on, the server issues have not returned and there are less incidents and calls during proctored exams. The procedures take some getting-used to, but it seems that the students learn quickly. For period 1, this means that incoming students need clear and comprehensive instructions (to be found on https://umlib.nl/proctoring) and that practice exams are essential.

It has also become clear that although using proctoring for large-scale exams is possible, it is to be used with caution for groups over 350 students. On the other hand, proctoring is a perfect customised solution for students who are not able to sit their exams on campus. Think of exchange students who need to take a resit after they return home, or athletes who are abroad for training or competition. Previously we did not have the means to facilitate this group, but now with proctoring, we can.

Most importantly, and even notwithstanding the many problems involved, proctoring has prevented a build-up of study delays by enabling exams that could not have taken place otherwise. It was a bumpy road, but in a time of crisis, we made the most of what was at hand.

Author:Tineke de Beaumont, coordinator digital exams at UM (DEXUM)

This article is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) License.

[et_bloom_inline optin_id=”optin_3″]