Improving the cost effectiveness of OSCEs

The value of using the objective structured clinical examination (OSCE) as a determinant of clinical skills is widely agreed. It is the preferred way to assess clinical skills across a variety of programmes, such as medical, pharmacy, veterinary and so on and as such, is considered a more robust approach to clinical skills assessment in comparison to the short or long case.  Its wide adoption across disciplines and international boundaries supports the agreement of its high utility, particularly in terms of reliability and validity.  However, for the vast majority of institutions, running OSCEs has a high associated cost – both directly and indirectly.

Where are the opportunities immprove OSCE cost effectiveness?

Given the pressure on education budgets, there certainly is a strong case for reducing these costs –making OSCEs more cost effective, whilst not compromising any aspects inherent in the assessment method, especially given the risks of passing an incompetent doctor for example. A study by Brown et al  in 2015 identified that it costs over £65,000 to run a high stakes OSCEs – this was a 15 station OSCE held over 2 days for 185 students. Adding in indirect costs this increases the cost to above £80,000.

The study identified that the large bulk of the costs were fixed due to the nature of the assessment – and these are the costs associated with examiner time on the day of the exam as well as those of standby examiners. The second most costly item is the non-examiner staff time required for the development, administration and production of exam tasks.

With the aid of robust exam management software that delivers eOSCEs, it is possible to reduce the non-examiner costs associated with running OSCEs – although of course the reduction in costs is going to be less due to the different pay scales associated with administrative staff compared with medical examiners. However, with the inclusion of opportunity cost associated with staff being able to focus on other tasks rather than exam administration, the savings will look more attractive – especially when considering that each School may run up to 5 OSCEs per year.

Reducing administrative time associated with OSCEs

So what functions should good exam management software perform that will reduce the amount of time administrative staff dedicate to running OSCEs?

• Exam preparation, review and approval and station authoring should be managed within the software. This eliminates the need for formatting of paper marking sheets, checking the marking pack of each examiner and the scanning time for processing thousands of paper OSCEs through an OMR scanner, as the software should manage it all. It also eliminates any manual rework of stations as authors should be able to make their own edits, rather than authors writing their edits as a word document or email and then administrative staff entering them in the software.
• Standard setting and exam and candidate reporting. With software eliminates any manual processes involved in getting exam results finalised and results given to candidates and course management.
• Emarking in an app or online reduces the chance of marking errors being made by examination staff. When there are no errors there is no need to review the candidates which failed or received a borderline grade as the software provides assurance that no marking oversights occurred. When marking exams electronically, it should not be possible to move onto another candidate until all the compulsory questions have been completed. This contrasts with running paper OSCEs where there usually is a post exam checking procedure to make sure marks haven’t been missed or overlooked, especially for borderline candidates.
• When managing the exam preparation and electronic delivery there will not be any costs associated with the printing of specialist marking sheets and their associated management. In the short term there is an additional investment in tablets, however the reduction in printing costs should overtime outweigh this.
• Production of candidate and examination reports should be done within the software, and be customisable to the institution’s needs. Being able to integrate candidate reports and results using an API into your own student portals should streamline administrative time even further.

Improving the cost effectiveness of OSCEs is just one consideration when choosing software to streamline OSCEs. Other reasons include strategic considerations such as aligning with digital strategies so that assessments reflect the real world scenarios as closely as possible for example, convenience, meeting student expectations for a digital exam taking and integrating with other systems and processes.

Take a look at this short video to get a snapshot of our exam software or give us a call to see how our exam software for OSCEs can improve the cost effectiveness of running OSCEs.