To SBA or VSA?…

…That is the question.

In medical education, there are a variety of assessment methods available for testing students. Historically, a popular approach is  to use Single Best Answer (SBA) exam questions, which have been used successfully in medical examinations for many years. However, recently we’ve seen a growing demand from customers for the Very Short Answer (VSA) question style as an alternative. In response, VSA’s are now available in the eSystem. Below, we’ve examined both of these question types in detail, exploring the benefits and drawbacks of each.

SBA questions (Single Best Answer)

With SBA exam questions candidates are asked to pick the single best answer from a list of answers presented alongside the question.

In medical assessments, good SBAs[1] typically consist of a short single question. This could include patient details, the medical complaint, any relevant history and observations, physical examination findings and any applicable results. The candidate reviews the information before selecting a single answer from the options available.  Often, there will be five answer options, however, it can be more or less.

Interestingly, this study [2] found that in most circumstances three options should be sufficient.  With there being a natural limit of plausible distractors, any more could be detrimental. It concludes that it’s the quality, not the quantity of the distractors which determines the quality of the assessment.

Single Best Answer questions enable a wide range of topics to be assessed with high reliability. They are considered to be more accurate in establishing candidate knowledge than true/false MCQs, where a student could just make a ‘lucky guess’.

As with true/false MCQs, SBA’s can be marked automatically, presenting a time-saving benefit and a distinct advantage over some other examination formats. Furthermore, since this question type has a correct answer, it eliminates any potential for examiner interpretation or bias.

Whilst there are some very obvious benefits to SBA questions, there are also some drawbacks. One of the most documented concerns is that they could be subject to cueing. A situation where the given answers could act as a prompt or clue to the candidate. The quality of the distractors is key. For example, if they are all implausible it will make the correct answer obvious by default and not because of the student’s knowledge in that area.

It’s also argued that SBA questions do not represent real-life scenarios since, in practice, doctors are not given a list of options when making diagnoses.

It’s these concerns that have prompted some to consider Very Short Answer (VSA) questions as an alternative means of assessment to SBAs.

VSA questions (Very Short Answer)

In a VSA question exam, students are given a scenario to which they have to generate their own answer. They are much like a short answer question, but with the limitation that the answer is restricted to only 2-3 words.

The advantage of this is that candidates must demonstrate their knowledge, which is more representative of real-life scenarios. VSA questions are not subject to cueing or guessing in the same way that SBA questions and true/false MCQs are.

Traditionally, a drawback of questions that require open text answers would be in the marking. These generally require examiners to review and mark the answers individually, rather than being marked automatically.

However, the use of exam software such as the Speedwell eSystem, coupled with the VSA question answers being limited to 2-3 words,  means that candidates can give their own answer and it can be marked automatically. Thus presenting the same advantage as SBA questions but without the cueing. Watch our short Video on VSAs in the eSystem to see how this works in practice.

Studies[3] have shown that Very Short Answer questions demonstrate high reliability and discrimination. In addition, cohorts generally scored lower on an exam with VSA questions than SBA equivalents. Suggesting that VSAs have a higher degree of validity and actually test student knowledge rather than exam proficiency.

The Perfect Exam?

While it’s clear that in some circumstances, certain question or assessment types present advantages over others, it’s also true that no single method is perfect. Ensuring that medical students are prepared for the real world of medical practice often requires a variety of examination types. With Speedwell’s eSystem software, you have the flexibility to choose the exam type to best suit your needs and gain all of the benefits of a streamlined exam process.

If you would like to find out more about our eSystem exam software, or you would like to arrange a personal, no-obligation demonstration, get in touch. Call our sales team on +44 (0) 1223 851703 or email info@speedwellsoftware.com

[1] Geeky Medics have produced a guide to writing effective SBA questions for Medical Examinations. Read the guide here

[2]  Kilgour, J.M., Tayyaba, S. An investigation into the optimal number of distractors in single-best answer exams. Adv in Health Sci Educ 21, 571–585 (2016). https://doi.org/10.1007/s10459-015-9652-7
[3] https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-016-0793-z (BMC Medical Education) – an example of a study