--- title: "Closing the loop: station redesign, teaching feedback and the annual review" description: "What happens after the OSCE quality report: which stations are revised or retired, what goes back to teaching, and what the annual review records." canonical: https://studydrome.com/docs/osce/quality/closing-the-loop/ updated: 2026-10-10 --- # Closing the loop: station redesign, teaching feedback and the annual review What happens after the OSCE quality report: which stations are revised or retired, what goes back to teaching, and what the annual review records. A quality report that produces no changes is a filing exercise. Closing the loop means three things. Every flagged station leaves the meeting with a verdict. Every teaching gap reaches the person who can fix it. And next year's report can show what happened. The AMEE guide on OSCE organization treats this as part of quality assurance rather than an optional extra. ## The loop in one paragraph The exam runs. The report says which stations behaved oddly. The committee decides what each one needs. Someone does the work before the next sitting. The next report shows whether it worked. Each step needs a name and a date, or the loop opens again at the same place. Most schools do the first two steps well and stop. The gap is nearly always between a finding and an owner. ## Revise, retire or keep Every flagged station gets one verdict. Recording it matters more than which one it is. **Keep and watch.** The metrics were odd once, and the cause is plausible: a small group at that station, a replaced examiner, a disrupted session. It runs again unchanged, and its numbers are compared next time. **Revise.** The station tests the right thing and scores it badly. This is the most common verdict and the one with the clearest playbook below. **Retire.** The station is exposed, has been used too often, or tests something the blueprint no longer needs. Retirement is a bank decision as much as a quality one, and the [station bank page](/docs/osce/design/station-bank-management/) covers rotation and exposure. The one verdict to avoid is silence. A station that remains in the bank without a decision attached will be used again. The next report will flag it again. ## What "revise" actually means AMEE Guide 49 records the redesign moves that followed metric review in practice. They are useful because they are specific. - [ ] Chunk several low-level criteria into one item, so the checklist stops rewarding trivia. - [ ] Add an intermediate grade descriptor, so the scale has a defined middle to land on. - [ ] Write three anchors for a scaled item rather than two, so the ends stop absorbing everything. - [ ] Make circuit layouts uniform so that candidates at the same station face the same conditions. - [ ] Refresh examiner training on the stations that showed weak agreement. - [ ] Run a dummy run of a revised station before it counts. - [ ] Give the revised station a new version number and record what changed and why. - [ ] Compare the revised station's metrics with its own previous values, not with the exam average. The first three are marking-scheme work and belong with the [marking scheme design](/docs/osce/design/designing-the-marking-scheme/). The fourth is a delivery decision. The fifth fixes stations nobody rewrote. A station with weak agreement is often fine, and its examiners were simply never calibrated on it. A revised station is a new station for analysis purposes. It goes back through [review and piloting](/docs/osce/design/station-review-and-piloting/) rather than straight into the next circuit. ## What goes back to teaching Some findings are not about the exam at all. A station with healthy metrics and a high failure count is the clearest case: the exam worked, and the cohort had not learned the material. Inside the assessment committee, that finding is worth almost nothing. In the hands of whoever owns the teaching, it is worth a great deal. Three practices make the handover real. Send the finding with the station, not just the number. The teacher can then see what candidates were asked to do. Send it while the cohort is still in the program. And ask for one-line feedback on what changed, which is what next year's report will check. The reverse handover matters too. When teaching changes, check the blueprint and the station set against it. Otherwise the exam drifts away from the curriculum it certifies. ## What examiners, candidates and patients tell you Numbers say where to look; people say why. The AMEE guide asks for evaluation from examiners and candidates as a standing part of the cycle. It also provides a station quality questionnaire, covering feasibility and validity, that a school can copy in shape. Three short instruments cover most of it. A station questionnaire for examiners, completed at the end of the session while it is fresh. A short candidate evaluation, kept separate from any complaint route. And the simulated patient debrief, which catches the things nobody else sees: a script that cannot be played consistently, a room that forces an odd position, a question candidates always misread. The guide lists six quality-assurance elements: standardization, external examiners, statistics, evaluation, examiner training, and peer review of stations. The report covers the third. This page is about the other five. ## The annual review Once a year, the committee reads the sittings together rather than one at a time. Four questions are enough. Which stations were flagged more than once, and what happened to them? Were the actions from last year completed? Are the exam-level numbers moving in a direction, and is the direction explained? And does the station bank still cover the blueprint after this year's retirements? The output is one page: what ran, what was found, what changed, what is outstanding. That page is also the cover sheet a review asks for, which is covered on [evidence for accreditors](/docs/osce/quality/evidence-for-accreditors/). ## Limits The loop is slow. Most changes can only be tested at the next sitting, which is a year away in many programs. A school gets few attempts, so change one thing at a time on any given station. It also depends on people who are not paid for it. Station revision is unglamorous work that lands on the same few academics. A committee that assigns it without protecting time will see the same stations flagged next year. The list of actions gets longer, and the names stay the same. ## Frequently asked questions ### What does closing the loop mean in OSCE quality assurance? Turning the findings of the quality report into decisions with owners and dates, doing the work before the next sitting, and checking in the next report whether it worked. Every flagged station gets one of three verdicts: keep and watch, revise, or retire. ### How do you fix a station with poor metrics? Usually by rewriting how it is scored. AMEE Guide 49 records chunking low-level criteria into single items, adding an intermediate grade descriptor, using three anchors instead of two, making circuit layouts uniform, refreshing examiner training, and running a dummy run before the station counts. ### When should an OSCE station be retired? When it is exposed, has been used too often, or tests something the blueprint no longer needs. Retirement is a station-bank decision as well as a quality one, so it is recorded with the bank's rotation plan and checked against blueprint coverage. ### What should be sent back to teaching after an OSCE? Any station with healthy metrics and a high failure count should be sent with the station itself so the teacher can see the task while the cohort is still in the program. Ask for one line back on what changed, and check it in next year's report. ### What should the annual OSCE review produce? One page naming what ran, what the reports found, what changed, and what is outstanding, plus answers to four questions: which stations were flagged repeatedly, whether last year's actions were done, whether exam-level numbers are moving, and whether the bank still covers the blueprint. ## Sources **Sources** 1. [Khan KZ, Gaunt K, Ramachandran S, Pushkar P. The Objective Structured Clinical Examination (OSCE): AMEE Guide No. 81. Part II: organisation and administration](https://doi.org/10.3109/0142159X.2013.818635) (Medical Teacher, 2013) 2. [Pell G, Fuller R, Homer M, Roberts T. How to measure the quality of the OSCE: a review of metrics. AMEE Guide No. 49](https://doi.org/10.3109/0142159X.2010.507716) (Medical Teacher, 2010) > [!TIP] > **In practice** > A loop that closes needs last year's numbers next to this year's, per station, without rebuilding a spreadsheet. To see what a platform keeps between sittings and what it does not, read /docs/osce/quality/in-studydrome-osce-analytics/.