With a vision to deliver excellence Newcastle University’s School of Medicine has high standards when it comes to teaching and assessment. When the Covid-19 pandemic hit, they needed a quick and robust solution to deliver remote online exams and as an existing Speedwell customer, they turned to us to help…

Newcastle University has a long history with medicine, with academics teaching the subject since 1834 in the founding School of Medicine and Surgery. Over the years a number of Colleges joined the School to form the University as we know it today. In 1984, the new Medical School was opened and further expanded in 2011 when Newcastle University Medicine Malaysia (NUMED) was established. The Medical School teaches over 2000 students on their MBBS programmes, with approximately a hundred of those being at NUMED.

Paper exam delivery

Newcastle University has been a Speedwell customer for over fifteen years, using Mulitquest and Clinquest software for question banking, exam marking and analysis, alongside OMR answer sheets and scanners to deliver paper exams.

Alice Morrison, the Learning and Teaching Assistant Manager (Assessment) at the University, explains that Speedwell is used for Single Best Answer (SBA), Short Answer Paper (SAP), Objective Structured Clinical Exam (OSCE), Written Skills, (WRISKE) and Modified Objective Structured Long Examination Review (MOSLER) exams within the Medical School.

Giving details on how the exams have historically been run, Alice explains that for SBA exams students are given an exam paper alongside a corresponding OMR sheet. After the exam, the sheets are collected and scanned for marking and analysis.

For the SAP and WRISKE exams students write their answer directly onto an exam paper, after the exam, the papers are marked in a process referred to as ‘team marking’. A group of markers are assigned several questions to collaboratively mark. They discuss each candidate’s answer and agree on a grade. This score is marked on an OMR sheet and then scanned so the results can be analysed. It’s quite a time-consuming process, typically taking several days to complete. For OSCE and MOSLER exams, examiners are given pre-prepared OMR sheets for each station which they use to mark each candidate.

Alice describes the huge amount of work involved in getting the sheets ready; “If you can imagine the OSCE’s for the Newcastle cohort, that’s 350 students, each doing 10 stations – that’s a lot of sheets to print, then each has to be sorted by candidate number, it’s very time-consuming.” Timetabling is complex too due to the large student numbers involved. It was a huge logistical challenge to ensure that everything ran smoothly and thoughts turned to how these processes could be improved.

The move to online exams

The University began discussing moving to online exams and in February 2020 they started a free trial of the eSystem. The team were keen to explore the administrative efficiencies they could gain, as well as minimising environmental impact and future-proofing their exam processes.

However, time constraints had meant that it hadn’t been a priority; “It’s the same old story” says Alice, “We were always too busy to get the time to focus on this properly” but when COVID-19 arrived, ascertaining how they could continue to run exams in a vastly changed world became urgent.

Since the eSystem trial was already in place, using it to run their early years exams during the pandemic seemed the logical step. Understanding the need to move quickly to ensure minimal impact on student learning, in April 2020, Alice approached Speedwell to ask if they could use the free trial site to run their upcoming exams.

Always willing to be as flexible as possible, and appreciating the challenges posed by the pandemic, Speedwell were happy to oblige and granted permission. Alice explains the first step was to migrate their existing question bank of SBA questions into the eSystem;

”Migrating our questions over to eSystem was easy to do. It was a huge migration, but Speedwell’s support team helped and ensured a seamless transition”

Next, it was over to the assessment team to create the exams. With the SBA questions already in the system, it was simply a case of selecting the relevant questions and creating the exam.

For the SAP exam, questions were created directly in the system, using the remote authoring function, so authors were able to do this from any location. Normally, candidates take exams on campus in exam halls. With lockdown and social distancing in place, this was not possible.

Instead, students would be taking exams from their home location on their own devices. This presented the team with a new hurdle to overcome; how to set the exam when there were candidates in different locations, spanning different time-zones. Alice recalls, “Thinking about the logistics created a bit of a headache! However, because we went online it was possible to do it, it was great and worked really well” She goes on to explain how they created different sub-sets of students by time zone in each year group.

Next, the exam was set up and cloned as many times as needed to effectively create a timed exam for each zone. Finally, they were able to link each sub-set of students accordingly. “eSystem actually made this surprisingly easy to do” Alice says. With this task complete the Medical School were in a position to run their first online exams in June – a matter of weeks from the initial enquiry in April.

The online experience

The first online exam was a SBA exam “Speedwell’s eSystem works brilliantly for our SBA exams” says Alice. She continues, “The student starts the exam, reads the question, selects their answer and in the background, the eSystem knows if it is correct or incorrect, so that’s very easy”

Post-exam it was clear to see where efficiencies had been gained; “Literally as soon as the exam is finished you have the results.” Additionally, Alice liked the fact that they were able to tell exam leads as soon as the exam was finished so they could immediately look into question performance.

For the SAP exam, a short written answer format, questions are presented to students with them typing their answer in the relevant box. When it came to marking, historically the marking teams would gather together and it was all paper-based. With eSystem, they used the online marking facility and gathered via Microsoft teams to collaborate. Alice says they were pleasantly surprised by how well it worked,

“The markers loved it! They found it much easier to use, it cut down on errors and it saved time”

It used to take the team 3+ days to complete the marking process which was reduced to just 2 days with the eSystem online marking. The marking team are very keen to use it again and the exam administrators are happy as there are no answer sheets to print. The student experience was largely positive too. Alice details how they set-up some practice sample questions so that all students had the opportunity to familiarise themselves with the system before the actual exam. She continues,

“The feedback was that students found the eSystem really easy to use. The fact that you could enlarge images as well was great and a feature they particularly liked”

Alice surmises, “It did work really well. It was easy to set up and very user-friendly, both for the assessment team and students. It was great that we could monitor students during the exam and it was simple if any students had an IT issue, to get them re-started and add on a few minutes if needed” For Alice and the team, perhaps the only downside was that some candidates were concerned about cheating. Despite mitigations such as randomising questions, timed exams and students signing declarations agreeing not to cheat, it was still a concern raised by a small minority. However, reassuringly, the University’s own investigations suggest that these concerns were unfounded.

Discussing the options available to secure exams further, Alice explains that remote proctoring solutions were discounted for a number of reasons. Safe exam browser, an eSystem feature, which completely locks down a device during an exam, was also ruled out as the University felt that it would not be appropriate to use in remote circumstances. Although, this approach is what Speedwell would recommend as best practice to ensure the integrity and security of invigilated on-campus exams.

The future

Ultimately, the fact that they were able to run exams at all in 2020 was a great relief for the team. Looking ahead, Alice would personally like to continue to use eSystem to run online exams on campus. The benefits gained in the speed of results and the marking process along with the administrative efficiencies are the main successes the team are looking to build on in the future. Alice reflects;

“In a way COVID did us a favour, it forced us to run online exams, and we’re pleased that it did, as it’s working well”