The Examiner Guide

Platform link:    https://abhs.speedwellesystem.com/esystem/

 From The Speedwell eSystem website:

INNOVATING OSCEs: How the Arab Board Set New Standards in Medical Certification

CASE STUDY: ARAB BOARD OF HEALTH SPECIALIZATIONS – DIGITAL OSCE

Recognising the need to modernise and standardise medical assessments across the Arab world, the Arab Board of Health Specializations set out to transition their clinical exams to a digital format. With a vision for a seamless, transparent and efficient process, they turned to Speedwell to help.

The Arab Board of Health Specializations is a well-respected institution dedicated to enhancing medical education across the Arab world. Founded in 1978, its mission is to ensure consistent, high standards for medical professionals in the region.

Core to this mission is ensuring that all assessments are conducted in a standard, fair, equal, and transparent manner, a responsibility overseen by Professor Sabeeh Mashhadani in his role as Assessment Director.

Assuming the role following a distinguished career as a consultant dermatologist, Professor Mashhadani also brought extensive experience in curriculum design and assessment from his tenure at Baghdad College of Medicine. This experience, coupled with his passion for technology, meant he was perfectly placed to innovate and modernise the exams.

SELECTING THE RIGHT PARTNER FOR THE ARAB BOARD’S DIGITAL OSCEs

Upon joining the Arab Board, one of the first things he noted was that the assessments were conducted using the traditional pen-and-paper method which had its limitations. Typically, there is one exam for all Arab Board candidates in a particular speciality, sometimes with the exam being delivered in different locations concurrently. He quickly identified that it was a challenge to administer standardised exams across multiple countries with this method.

Additionally, there were issues such as a lack of centrally stored questions, no historical data for improving future exams and inconsistent testing environments. He further noted that providing candidate feedback was cumbersome and time-consuming, highlighting further inefficiencies with this traditional approach.

Professor Mashhadani knew that change was necessary. He recognised the benefits of online exams and has been instrumental in championing the transition to digital assessments. The goal being to streamline operations and enhance the overall efficiency and quality of the examination processes of the Arab Board.

NAVIGATING THE TRANSITION – THE PATH TO DIGITAL OSCEs

The digital transformation of exams started with the implementation of online written knowledge exams. Keen to build on this, attention soon shifted to how to utilise the latest examination technologies to enhance OSCE assessments.

However, identifying the right partner for this task proved challenging. Many online examination platforms lacked the capability to manage the intricate requirements of OSCE exams.

After evaluating options, Professor Mashhadani knew Speedwell would be the right choice to help him deliver his vision:

“Speedwell’s eSystem offers a complete, easy-to-implement solution that’s intuitive and user-friendly. It provides exactly what we need to transform our OSCE assessments.”

Not only is the eSystem a web-based platform that requires no local installation, but Speedwell also brought extensive experience in the medical education sector. With a proven track record of helping institutions successfully run OSCEs in various formats—on paper, online, and offline over many years—Speedwell demonstrated a deep understanding of the complexities associated with practical clinical exams. This proven expertise and experience provided Professor Mashhadani with the confidence that the eSystem was the right choice to modernise their assessments.

Furthermore, the software offers a robust solution that functions reliably with or without Internet connectivity. The latter was particularly important to Professor Mashhadani and his team as connections can be unreliable in parts of the Arab region at times.

TRAINING FOR CHANGE – EMBRACING DIGITAL OSCEs

Despite having confidence in their chosen partner, the path to delivering online OSCEs was not without challenges.

Introducing a new assessment format typically generates considerable concern among stakeholders, with resistance to change being a primary obstacle, and the Arab Board was no exception. Professor Mashhadani believes that a fear of technology also played a significant role in his experience.

He explains that many of those involved belonged to a generation that did not grow up with technology. Understandably, there were reservations about learning to use new systems, coupled with a general lack of awareness about the benefits of transitioning to digital OSCEs. Additionally, there were concerns about the reliability of digital systems and their ability to maintain the integrity and rigour of the examinations.

Addressing these concerns required a strategic approach involving comprehensive training, a robust support system and clear communication of the advantages to build confidence in the new format.

The Arab Board comprises more than 40 scientific councils, covering fields such as dermatology, general medicine, and surgery. Training such a large number of professionals initially seemed daunting. During early discussions, there were understandable concerns from stakeholders about the perceived complexity of the new system and the potential learning curve.

However, Professor Mashhadani was determined and with support and comprehensive training from Speedwell, he became an expert in the software. He personally undertook the training of all his colleagues, demonstrating the eSystem’s ease of use at each stage.

Key to his success was understanding his audience and teaching them in a manner that made them feel comfortable. He explains he focused on simplifying the training, concentrating on core areas relevant to the individual’s role to avoid overwhelming them with the system’s full capabilities.

“Even the examiners who did not think they could learn the software, like using the eSystem once they use it.”

RUNNING THE FIRST DIGITAL OSCE

The General Surgery Council led the way with the transition to digital OSCEs. Beginning with a successful pilot that provided confidence to proceed to a full-scale exam. It was decided that the exam would be delivered offline, with Professor Mashhadani assisting with the station set-up as well as monitoring the exam as it was in progress. It was delivered successfully and everyone was delighted with how well the exam went:

“We’ve achieved remarkable results with our first digital OSCE – quick, instant results and amazing statistical insights. It’s created a new standard for our exams.”

Now, using the eSystem, they’re amazed that they can instantly complete post-exam statistical calculations with the click of a button. Furthermore, decisions can be made about the test immediately after the exam, which is a huge tangible difference.

When it comes to candidate feedback, the eSystem is delivering substantial benefits too. Previously, they could only provide feedback for each station; it wasn’t feasible to offer detailed insights. Now, they can offer two types of feedback: station-specific and competency-based.

“Sometimes a candidate might pass a station but consistently underperform in a particular area, such as differential diagnosis. The eSystem enables us to identify and remedy any common areas of weakness.”

IMPLEMENTATION OF THE ARAB BOARD DIGITAL OSCE MANAGEMENT SYSTEM

The Family Medicine Council delivered the clinical exam simultaneously in test centres in different countries. However, historically, there was no connection between the centres—they would all conduct the exams independently with no monitoring between them.

In collaboration with the Dubai Health Authority and the General Secretariat of the Arab Board, a clinical examination was conducted using Speedwell’s eSystem for 126 doctors across five accredited test centres in various Arab countries. This exam, monitored through a centralised control centre, was seamlessly administered with a high level of transparency and was deemed a huge success.

SETTING A NEW STANDARD: LEADING THE FUTURE OF MEDICAL EDUCATION

Professor Mashhadani’s vision extends beyond the current achievements. He aims to expand the Digital OSCE Management System to more councils within the Arab Board, hoping to have 12 scientific councils using it by the end of 2024 and the remaining councils the following year.

“It’s not a choice—if we do not go through the digital transformation, we will not achieve our goals.”

Together, the Arab Board and Speedwell have set a new benchmark for medical certification in the Arab region, ensuring that the next generation of healthcare professionals are evaluated with precision, fairness, and efficiency.

Interpreting Data Analysis Results: Practical Examples