Skip to content

Standard setting in StudyDrome Exam Manager

Four methods, one cut score you can defend

StudyDrome Exam Manager sets OSCE pass marks with four methods: Custom, Angoff, Borderline Group, and Borderline Regression. Each method derives a mark for every station. You can lock a station's mark, make it must-pass, or withdraw it. An exam can also require a minimum number of stations passed.

Book a pilot

Which standard-setting methods does StudyDrome support?

Method

How the station pass mark is derived

Custom

A preset pass mark you type in.

Angoff

Expert-judgment pass marks, entered directly against each station.

Borderline Group

The mean or median checklist score of candidates rated at the borderline grade.

Borderline Regression

Checklist scores regressed on Global Rating. The mark is the fitted value at the borderline grade.

Each method also determines how station marks are converted into an exam mark. Custom, Borderline Group and Borderline Regression sum the stations. Angoff offers Sum or Average, and defaults to Average. Borderline Group offers Mean or Median, and defaults to Mean.

Borderline Regression is ordinary least squares, fitted station by station. It returns slope, intercept, and R² to four decimal places. The derived mark is clamped within the achievable range and rounded to 2 decimal places.

For the methods themselves, read OSCE standard-setting methods compared or try the OSCE standard-setting calculator.

How do you calculate a pass mark on screen?

  1. 1

    Open the OSCE exam wizard and go to step 9, Standard Setting. The step lists every station in the circuit.

  2. 2

    Choose the method: Custom, Angoff, Borderline Regression, or Borderline Group.

  3. 3

    Set the calculation method where the method is offered. Angoff takes Sum or Average. Borderline Group takes Mean or Median.

  4. 4

    Press Calculate. Every station gets its own row, with the pass mark the method derives.

  5. 5

    Read the row: Code, Title, n, Lowest, Highest, Mean, Std Dev, R², Borderline Grade, Score, Pass Mark and Passed. Hover over n for the number of candidates assessed at that station.

  6. 6

    Adjust station by station with the Lock, Must Pass, and Withdrawn switches. Press Reset to unlock every station and clear the marks.

What can you control at each station?

Every station carries its own standard-setting row. The controls are independent, so one station can be locked while the rest recalculate:

  • Borderline grade:Each station holds its own borderline-grade anchor, and it can override the exam-level grade.

  • Pass mark:Each station holds its own pass mark, shown in the Pass Mark column beside the station statistics.

  • Lock:A locked pass mark is used verbatim. Recalculating the exam leaves that station exactly where the panel put it.

  • Must Pass:Scoring below a must-pass station's mark fails the exam, whatever the total.

  • Withdrawn:A withdrawn station is left out of every exam-mark sum and average. The row notes "Station withdrawn from calculations".

  • Excluded from Global Rating:Rest stations carry this flag permanently. A borderline method notes "Station is excluded from the Global Rating" and moves on.

How does an exam decide whether to pass or fail?

Three conditions decide the result. They are evaluated together, and every one that applies must hold.

Condition

What it checks

Exam pass mark

The candidate's total reaches the exam pass mark.

Minimum stations

The candidate passes at least the minimum number of stations when that rule is on.

Must-pass stations

The candidate passes every station marked Must Pass.

A station counts toward stations passed when it has a pass mark and is not withdrawn. Switch the minimum-stations rule off, and the count clears itself.

The exam pass mark is determined by the stations, per the method's own rules. You can also set it yourself. Turn on Override exam pass mark, and the screen keeps the derived figure in view, labeled Overridden to. The pass rate is reported to two decimal places.

These rules belong to the circuit. OSCE exams in StudyDrome cover how that circuit is built and delivered.

Which warnings appear before you publish a cut score?

Four named warnings are raised during the calculation. Each one appears as a badge on the station it belongs to, with the full sentence behind it:

  • Same rating for all:Every candidate received the same Global Rating, so the regression is undefined for this station. Consider the Borderline Group method.

  • Negative slope:Checklist scores decrease as Global Ratings increase. Review the station checklist and examiner scoring.

  • Low R²:R² is below 0.5, so the checklist and the examiners' global judgment agree poorly at this station.

  • Small cohort (n<20):Fewer than 20 completed candidates, so the regression result may be unstable.

Where do the R² and cohort thresholds come from?

Both thresholds are published, not invented.

R² below 0.5 flags poor agreement between the checklist and examiner judgment. That threshold follows Pell and Roberts, AMEE Guide 49. The regression report carries the same reference in its methodology block.

A cohort under 20 completed candidates flags a potentially unstable result. Read that as a conservative floor rather than a sufficiency test. The resampling literature places the onset of serious instability later, and later work has found that the spread of global grades predicts stability better than headcount does. Our guide to borderline regression sample size works through the evidence and the per-station checks.

Borderline regression fits a line from 3 valid candidates upward, and reports the count it found. A valid candidate is one who completed the station and received a Global Rating. For the method in plain terms, see what the borderline regression method is.

The warnings are advisory. They surface before you publish a cut score, and the decision stays with the assessment lead.

What a small cohort does to the pass mark

Standard error of the borderline-regression pass mark at the two extremes of the cohort sizes tested.

Over 200 candidates15 candidates

Whole-exam pass mark

Under 0.5%
About 1.7%

One station's pass mark

1–2%
4–10%
Source: Homer, Pell, Fuller & Patterson (2016), Medical Teacher 38(2):181–188 — 1,000 resamples per cohort size across four real OSCE data sets.

Where does standard setting apply?

  • Standard setting applies to OSCE circuits. Written papers are graded on grading scales, with difficulty-weighted scoring where you want it.

  • Angoff pass marks are entered directly against each station by whoever holds the judgment.

  • Borderline Group averages the checklist scores of candidates whose Global Rating equals the borderline grade. Mean or median, your choice.

  • Borderline Regression is ordinary least squares on one predictor. Slope, intercept, and R² are returned, so you can read the fit yourself.

  • Borderline methods read the examiners' Global Ratings, so the exam carries a Global Rating Scale. The screen says so when one is still to be attached.

  • Minimum stations to pass and must-pass are circuit rules. They act on the station results the rotation produces.

  • Locked marks survive recalculation. A panel decision, therefore, holds while the rest of the circuit is refitted using the data.

Who needs standard-setting software?

A cut score is the number a candidate appeals to. These are the people who have to stand behind it.

  • Assessment leads who choose the method must then explain their choice to an exam board.

  • OSCE convenors who need one weak station handled without moving the rest of the circuit.

  • Quality and appeals panels, who ask how the pass mark was derived and from which candidates.

  • Psychometricians who want slope, intercept, and R² per station rather than a single headline number.

Frequently asked questions

Can we lock a station's pass mark so recalculation leaves it alone?

Yes. Every station row carries a Lock switch. A locked station keeps its pass mark verbatim, and a recalculation refits the other stations around it. That lets a panel settle one station by judgment and derive the rest from the data—Press Reset to unlock every station and clear the marks again.

How does a must-pass station change the exam result?

A must-pass station is a hard gate. A candidate who scores below its pass mark fails the exam, regardless of the total. StudyDrome evaluates the conditions together: the total against the exam pass mark, the minimum number of stations passed, and every must-pass station. All of them must hold before a candidate passes.

What happens when a cohort is small?

Borderline regression fits a line per station, so it needs data. It runs from 3 valid candidates upward, and reports the count it found. Below 20 completed candidates, a small-cohort warning is raised. That warning is advisory, so your exam board keeps the decision. The count is per station, which matters when parallel circuits split one cohort across several copies of the same station.

Can we override the calculated exam pass mark?

Yes. Turn on Override exam pass mark and enter the figure your board agreed. The screen keeps the derived value in view, labeled Overridden to, so the calculation and the decision sit side by side. Switch the override off, and the exam pass mark returns to the value derived from the stations.

What happens to a station we withdraw from the calculation?

A withdrawn station drops out of every exam-mark sum and average, and its row notes that it was withdrawn from calculations. It also stops counting toward stations passed. The station and its scores stay on the record, so the circuit you ran and the marks you set remain readable side by side afterward.

How does StudyDrome grade written exams?

Written exams are graded on grading scales, with difficulty-weighted scoring where you want it. Standard setting applies to OSCE circuits, in which each station sets its own pass mark based on checklist scores and Global Ratings. Each exam type, therefore, uses the mechanism that fits it, and both are published as frozen snapshots that feed into item analysis.

Book a pilot

Bring one circuit and last year's scores. We will run all four methods over them and show you the station marks each one produces.

Book a pilot

Related