Using borderline regression for standard setting

Using Borderline Regression for standard-setting

Hi, I’m Trish from Speedwell. In this clip I will be showing how to set a pass mark using borderline regression method.

When marking, the examiner will mark each objective task using Yes/No as that is how the station was set up. For the overall performance, the overall grade or subjective mark, we have pass/borderline/fail – this student receives a borderline, and the examiner can add any comments in the optional box below.

Switching to the esystem borderline regression report we can see that this station does not have a calculated or user-defined pass mark.  Below we see the subjective overall grade scale set for this station (pass/borderline/fail) and the corresponding scores.  We can also see the mean score for students achieving each grade and how many students received each grade on the right

In the graph below we see the objective score plotted on the Y-axis against the overall grade scores (fail 0, borderline 1 and pass 2) plotted on the X-axis. The regression line is plotted here And each plot shows the number of students.  Here we have 2 students who passed receiving an objective score of 15.

Back at the exam, I want to set my standard. From the dropdown, I have a choice of setting a custom pass mark, and as well as the borderline regression, I also have options for using Angoff method, or Borderline group method.

When I choose borderline regression, I get an extra option in the right for a minimum number of stations to pass and as we are setting the exam pass mark this is shown in red.  I could override that pass mark once I’ve calculated it.

The station details are listed below.  And for each station, I can see the lowest, highest, the mean average, the std deviation and the R squared.

I could manually set my grade for each station and the score would show next to it. I could manually enter a score or pass mark here.  I could lock my settings, set a must pass for a station and there is an option to withdraw a station if I need to.

When I click Calculate, we can see the exam pass mark is calculated above. the esystem has automatically set the borderline from the overall grade. Along with the corresponding score, which is one, and it’s used that linear regression from the graph to calculate the pass mark and it shows how many students have passed with a borderline for each of the stations,

I could manually adjust the pass mark for each of the stations like this, and we can see how that affects the pass rate and the exam pass mark up here.

I might need to switch on a must-pass or an essential pass station, like this.

And I could choose to override any exam pass mark I’ve set as well using the option in this box.

Once I have set all my options on the borderline regression standard-setting screen, I save the exam settings.

Back in the borderline regression report, we can see the calculated pass mark now. And our user-defined adjustment. Based on our use of borderline for the overall grade. So our selection of borderline, or one on the x-axis, shows here on the graph, and where that line intersects with the regression line, our pass mark is calculated.  We rounded that to 12 and that user-defined pass mark shows as well

And there is one of borderline regression chart for every station in our exam – clearly showing how we set our standard for each station.

In our candidate performance report, we can see the mean score, the standard deviation, the number of candidates and the overall pass mark we have set. Along with any candidate details we have chosen to display, and the scores for each candidate

The first candidate receives 14 which is a fail.  The second candidate received 27 but failed

The candidates s receiving 29 25 and 30 all passed

9 points is a fail,24 and 31 points are passes.

The candidate receiving 27 who failed, failed a must pass in our exam but this could equally have been failed to pass a minimum number of stations, had we set that.

For more information about standard-setting or to book a free demonstration please contact us at speedwell software.com