Standard setting in StudyDrome Exam Manager

Four methods, one cut score you can defend

StudyDrome Exam Manager sets OSCE pass marks with four methods: Custom, Angoff, Borderline Group and Borderline Regression. Each method derives a mark for every station. You can lock a station's mark, make it must-pass, or withdraw it. An exam can also require a minimum number of stations passed.

Book a pilot

Which standard-setting methods does StudyDrome support?

Method

How the station pass mark is derived

Custom

A preset pass mark you type in.

Angoff

Expert-judgment pass marks, entered directly against each station.

Borderline Group

The mean or median checklist score of candidates rated at the borderline grade.

Borderline Regression

Checklist scores regressed on Global Rating. The mark is the fitted value at the borderline grade.

Each method also decides how station marks become an exam mark. Custom, Borderline Group and Borderline Regression sum the stations. Angoff offers Sum or Average, and defaults to Average. Borderline Group offers Mean or Median, and defaults to Mean.

Borderline Regression is ordinary least squares, fitted station by station. It returns slope, intercept and R² to four decimal places. The derived mark is clamped into the achievable range and rounded to two decimals.

For the methods themselves, read OSCE standard-setting methods compared or try the OSCE standard-setting calculator.

How do you calculate a pass mark on screen?

  1. 1
    Open the OSCE exam wizard and go to step 9, Standard Setting. The step lists every station in the circuit.
  2. 2
    Choose the method: Custom, Angoff, Borderline Regression or Borderline Group.
  3. 3
    Set the calculation method where the method offers one. Angoff takes Sum or Average. Borderline Group takes Mean or Median.
  4. 4
    Press Calculate. Every station gets its own row, with the pass mark the method derives.
  5. 5
    Read the row: Code, Title, n, Lowest, Highest, Mean, Std Dev, R², Borderline Grade, Score, Pass Mark and Passed. Hover n for the number of candidates assessed at that station.
  6. 6
    Adjust station by station with the Lock, Must Pass and Withdrawn switches. Press Reset to unlock every station and clear the marks.

What can you control on each station?

Every station carries its own standard-setting row. The controls are independent, so one station can be locked while the rest recalculate:

  • Borderline grade: each station holds its own borderline-grade anchor, and it can override the exam-level grade.
  • Pass mark: each station holds its own pass mark, shown in the Pass Mark column beside the station statistics.
  • Lock: a locked pass mark is used verbatim. Recalculating the exam leaves that station exactly where the panel put it.
  • Must Pass: scoring below a must-pass station's mark fails the exam, whatever the total.
  • Withdrawn: a withdrawn station is left out of every exam-mark sum and average. The row notes "Station withdrawn from calculations".
  • Excluded from Global Rating: rest stations carry this flag permanently. A borderline method notes "Station is excluded from the Global Rating" and moves on.

How does an exam decide pass or fail?

Three conditions decide the result. They are evaluated together, and every one that applies must hold.

Condition

What it checks

Exam pass mark

The candidate's total reaches the exam pass mark.

Minimum stations

The candidate passes at least the minimum number of stations, when that rule is on.

Must-pass stations

The candidate passes every station marked Must Pass.

A station counts toward stations passed when it has a pass mark and is not withdrawn. Switch the minimum-stations rule off and the count clears itself.

The exam pass mark comes from the stations by the method's own rule. You can also set it yourself. Turn on Override exam pass mark and the screen keeps the derived figure in view, labelled Overridden to. Pass rate is reported to two decimals.

These rules belong to the circuit. OSCE exams in StudyDrome covers how that circuit is built and delivered.

Which warnings appear before you publish a cut score?

Four named warnings are raised while the calculation runs. Each one appears as a badge on the station it belongs to, with the full sentence behind it:

  • Same rating for all: every candidate received the same Global Rating, so the regression is undefined for this station. Consider the Borderline Group method.
  • Negative slope: checklist scores decrease as Global Ratings increase. Review the station checklist and examiner scoring.
  • Low R²: R² is below 0.5, so the checklist and the examiners' global judgment agree poorly at this station.
  • Small cohort (n<20): fewer than 20 completed candidates, so the regression result may be unstable.

Where do the R² and cohort thresholds come from?

Both thresholds are published, not invented.

R² below 0.5 flags poor agreement between the checklist and examiner judgment. That threshold follows Pell and Roberts, AMEE Guide 49. The regression report carries the same reference in its methodology block.

A cohort under 20 completed candidates flags a result that may be unstable. That threshold follows Homer et al. 2015 and Moreno-López et al. 2022.

Borderline regression fits a line from 3 valid candidates upward, and reports the count it found. A valid candidate is one who completed the station and received a Global Rating. For the method in plain terms, see what the borderline regression method is.

The warnings are advisory. They surface before you publish a cut score, and the decision stays with the assessment lead.

Where does standard setting apply?

  • Standard setting applies to OSCE circuits. Written papers are graded on grading scales, with difficulty-weighted scoring where you want it.
  • Angoff pass marks are entered directly against each station, by whoever holds the judgment.
  • Borderline Group averages the checklist scores of candidates whose Global Rating equals the borderline grade. Mean or median, your choice.
  • Borderline Regression is ordinary least squares on one predictor. Slope, intercept and R² are returned, so you can read the fit yourself.
  • Borderline methods read the examiners' Global Ratings, so the exam carries a Global Rating Scale. The screen says so when one is still to be attached.
  • Minimum stations to pass and Must Pass are circuit rules. They act on the station results the rotation produces.
  • Locked marks survive recalculation. A panel decision therefore holds while the rest of the circuit is refitted from the data.

Who needs standard-setting software?

A cut score is the number a candidate appeals. These are the people who have to stand behind it.

  • Assessment leads who choose the method, then have to explain the choice to an exam board.
  • OSCE convenors who need one weak station handled without moving the rest of the circuit.
  • Quality and appeals panels, who ask how the pass mark was derived and from which candidates.
  • Psychometricians who want slope, intercept and R² per station rather than a single headline number.

Frequently asked questions

Can we lock a station's pass mark so recalculation leaves it alone?

Yes. Every station row carries a Lock switch. A locked station keeps its pass mark verbatim, and a recalculation refits the other stations around it. That lets a panel settle one station by judgment and derive the rest from the data. Press Reset to unlock every station and clear the marks again.

How does a must-pass station change the exam result?

A must-pass station is a hard gate. A candidate who scores below its pass mark fails the exam, whatever the total. StudyDrome evaluates the conditions together: the total against the exam pass mark, the minimum number of stations passed, and every must-pass station. All of them must hold before a candidate passes.

How many candidates does borderline regression need?

Borderline regression fits a line per station, so it needs data. It runs from 3 valid candidates upward, and reports the count it found. Below 20 completed candidates it raises a small-cohort warning, citing Homer et al. 2015 and Moreno-López et al. 2022. That warning is advisory, so your exam board keeps the decision.

Can we override the exam pass mark that the calculation produces?

Yes. Turn on Override exam pass mark and enter the figure your board agreed. The screen keeps the derived value in view, labelled Overridden to, so the calculation and the decision sit side by side. Switch the override off and the exam pass mark returns to the value the method derives from the stations.

What happens to a station we withdraw from the calculation?

A withdrawn station drops out of every exam-mark sum and average, and its row notes that it was withdrawn from calculations. It also stops counting toward stations passed. The station and its scores stay on the record, so the circuit you ran and the marks you set remain readable side by side afterwards.

How does StudyDrome grade written exams?

Written exams are graded on grading scales, with difficulty-weighted scoring where you want it. Standard setting applies to OSCE circuits, where each station produces its own pass mark from checklist scores and Global Ratings. Each exam type therefore uses the mechanism that fits it, and both publish as frozen snapshots that feed item analysis.

Book a pilot

Bring one circuit and last year's scores. We will run all four methods over them and show you the station marks each one produces.

Book a pilot

Related