An OSCE needs seven policies in writing before its first sitting: the pass rules, the critical-error rule, retakes, the exam board's remit, appeal grounds and windows, mitigating circumstances, and misconduct. Each one is short. Each one is approved by the committee before the blueprint is signed. This page lists what every policy must state and who approves it.
Why write the policies before the exam?
Every OSCE produces a small number of hard cases. A candidate fails by one mark. A candidate passes every station, but one, and that one was a critical error. A simulated patient went off script. A device lost its connection for two minutes. Each case needs a rule. A rule written after the case is a rule written for that candidate, and it will not survive an appeal.
The AMEE guide on OSCE organization lists the after-exam work in the following order: the standard is set, the results are ratified by the exam board, the results are published, complaints and appeals are handled, and external examiners report. Every step in that list runs on an existing policy. The Georgian accreditation review of medical schools makes the same point from the regulator's side. Among its recommendations was that "the regulation governing retakes" be brought "in line with the legislation". A school whose retake rule contradicts national law finds out at accreditation, or in court.
Which rules decide who passes?
The pass rules are the heart of the regulations. Four decisions belong in them.
The standard-setting method. The AMEE guide notes that norm referencing is "usually deemed unacceptable for clinical competency licensing tests". The method is criterion-based, chosen before the exam and named in the regulations. The scoring section compares the methods.
Compensation or conjunction. A compensatory rule lets a strong station make up for a weak one: the candidate passes on the total. A conjunctive rule adds a second hurdle, such as a minimum number of stations passed. Homer and colleagues set out why and how to use a stations-passed rule. Whichever you choose, write it down with its numbers.
Critical errors. Some actions fail a station, whatever else the candidate did: a dangerous drug dose or a missed safety check. The rule states what counts, who confirms it, and whether it fails the station or the exam. The station design section covers the design.
The borderline zone. What happens to a candidate within a small margin of the pass mark. Options include automatic review of the station data, a second look at the examiner's global rating, or nothing. "Nothing" is a valid rule. Silence is not.
What does the exam board do?
The assessment committee plans the exam. The exam board owns the result. In many schools, the same people sit on both, but the board meets with a quorum, a chair who did not run the exam, minutes, and often an external examiner. Its remit:
- Receive the analysis. Reliability, station metrics, examiner comparison. The board report post shows what that pack looks like.
- Ratify the standard. The pass mark is proposed by the analyst using the named method and approved by the board.
- Ratify the results. Individual outcomes, including any station removed after the analysis.
- Decide the hard cases. Borderline candidates, critical-error confirmations, mitigating circumstances.
- Hear appeals or delegate them to a panel that reports back.
- Record the external examiner's report and the school's response.
The board's minutes are the evidence an accreditor asks for. The Georgian review found that experts "could not find evidence related to reliability and validity" of the assessment methods at most schools, and that "only one institution applies the psychometric approach". A board that receives the analysis and minutes its decisions has that evidence by default.
The policy checklist
The table lists each policy, what it must state, and who approves it. Write them in this order; each one references the ones above it.
Policy | Must state | Approved by |
|---|---|---|
Pass rules | Standard-setting method; compensatory or conjunctive rule with numbers; borderline handling | Committee, then the academic body |
Critical errors | What counts; who confirms; station or exam consequence; how candidates are told | Committee |
Retakes and resits | Number of attempts; timing; whether the whole exam or failed stations are resat; the cap on the resit result; alignment with national law | Academic body |
Exam board | Membership; quorum; chair; external examiner; what it ratifies; minutes | Academic body |
Appeals | Grounds; window; who hears it; possible outcomes; what evidence the candidate can see | Academic body |
Mitigating circumstances | What counts; evidence; deadline; whether a result is voided or a resit granted | Committee |
Misconduct | What counts in an OSCE, including sharing station content; process; penalties | Academic body |
Reasonable adjustments | Who applies; deadline; what can be adjusted; what cannot | Committee |
Consent, recording and data | Who is recorded; purpose; retention; access for appeals | Committee, with counsel |
The last two rows have their own pages: accessibility and reasonable adjustments and consent, recording and data protection.
What are valid grounds for an OSCE appeal?
Most institutions allow appeals on process and not on judgment. The three usual grounds:
- Procedural irregularity. The exam was not run as the regulations describe. A timing signal failed. A station's instructions were missing. The adjustment approved for the candidate was not applied.
- Bias or conflict of interest. An examiner with a personal connection to the candidate marked the station, against the code of conduct.
- Mitigating circumstances not considered. Evidence submitted in time was not put before the board.
Disagreement with an examiner's professional judgment is not a ground. Write that sentence into the policy, because it is the ground most candidates will try. Then state the window, usually two to four weeks from the release of results, who hears the appeal, and the outcomes: no change, a resit at the next sitting, or a review of the station data by the board.
The recording and consent policy matters here. If stations are recorded, the appeal policy should say whether the candidate can see the recording and who else can. If they are not recorded, the appeal rests on the mark sheets, the examiner's notes and the floor manager's incident log. All three need to exist.
How should retakes be regulated?
Retake rules cause more regulatory trouble than any other policy, because they touch national law on progression. Decide five things:
- How many attempts does a candidate have, including the first?
- Whether the resit covers the whole exam or only failed stations. Some schools resit the complete exam; some resit a second set of stations under a sequential design. Whichever you pick, the resit must sample the blueprint well enough to be reliable on its own.
- When the resit happens and what support is offered before it.
- Whether a resit result is capped, and how it appears on the transcript.
- What happens after the final failed attempt.
Then check the five against the national regulations on student progression. The Georgian review's recommendation on retakes was issued because at least one school's rule did not align with the law. That check takes an afternoon with the regulations open. Skipping it costs a year at accreditation.
Frequently asked questions
What policies does an OSCE need before it runs?
Seven at minimum: pass rules with the standard-setting method, a critical-error rule, retakes, the exam board's remit, appeals, mitigating circumstances, and misconduct. Add the reasonable adjustment policy and the consent and recording policy. All are approved before the blueprint is signed, because the pass rules shape how stations are scored.
What can a candidate appeal in an OSCE?
In most institutions, the process, not the judgment. Valid grounds are a procedural irregularity, bias or a conflict of interest, and mitigating circumstances that were not considered. Disagreeing with an examiner's professional judgment is not a ground, and the policy should say so in those words.
Who sets the OSCE pass mark, the committee or the exam board?
The analyst proposes it using the method named in the regulations. The exam board ratifies it, with the reliability and station data presented to it. The committee chose the method months earlier. Keeping the proposal, the method, and the ratification in different hands is what makes the standard defensible.
Should a failed OSCE be resat in full or by station?
Either can be defended if the regulations specify which, and the rest is reliable on its own. A full resit is simpler and samples the blueprint again. A station-only resit needs enough stations to give a stable result. Check the choice against national progression rules before it is published.
Does an OSCE need an external examiner?
Many regulators expect one for high-stakes exams and the AMEE guide lists the external examiner's report among the after-exam steps. The external examiner reviews the stations, observes the day, sees the analysis and reports to the board. The board minutes the report and its response, which is the evidence accreditors ask for.