An exam assembly strategy decides where each question comes from: a fixed paper every candidate sits, or a fresh sample drawn from your question banks. StudyDrome Exam Manager supports three bank sampling modes and two independent randomization settings. The choice is not about security alone. It changes what your item statistics can tell you afterward.
That second consequence is the one exam offices meet late. Randomization usually arrives as an anti-cheating measure. It is also a measurement decision. It quietly sets the ceiling on what you can prove later.
What are the three ways to assemble an exam?
Every assembly strategy is one of three shapes. The names differ between platforms. The shapes do not.
Strategy | How the paper is built | Best when |
|---|---|---|
Fixed form | You name every question. Everyone sits the same paper | Comparability matters most, or you are standard setting on this paper |
Per-bank quota | You pick banks and set how much each contributes, by share or by exact count | You need topic balance guaranteed on every draw |
Free draw | You pick banks and let the system fill the paper from all of them | The banks are already balanced and deep |
A paper does not have to be purely one shape. You can name some questions and let the system draw the rest. One paper, in whatever order you choose.
One practical note on quotas. If you set shares as percentages, they must add up to 100%. That check belongs at publish, not as you type. A paper whose shares total 95% should be refused outright, never quietly delivered nine items short.
When is a fixed form the right choice?
Three cases, and none of them is about security.
You are setting the standard on this paper. An Angoff panel rates each item on the paper in front of them. Draw the paper at random per candidate, and the items the judges rated are not the items anyone sat. The cut score no longer refers to the exam. Borderline regression behaves differently. It works from candidate performance, not item judgments. But any item-by-item method needs a fixed paper.
You need item statistics you can act on. Every item needs enough responses before its difficulty and discrimination have any meaning. Split 200 candidates across a 500-item bank, and most items collect a handful of answers each. A discrimination index built on a handful of responses is noise. A sensible system withholds the verdict rather than publishing one. A fixed form concentrates responses where you can read them. Assembly and item analysis are the same decision seen from two ends.
Comparability is what you will be asked to defend. One paper, one score scale. Nobody can argue that a candidate drew a harder set.
What does randomization actually protect against?
Three separate switches get called "randomization". They defeat three different things. Turning them all on because they sound safe makes an exam harder to defend, not easier.
- Question order. Defeats copying from a neighbor's screen, and messages about "question seven". Does nothing about a leaked bank.
- Answer-option order. Defeats a shared answer key — the "C, A, D, B" list passed between sittings. Does nothing if the questions themselves circulate.
- Which questions are drawn. Devalues a leaked paper, because there is no single paper to leak. Does nothing about someone sitting beside the candidate.
A useful middle setting on question order gets overlooked. Rather than shuffling the whole paper, you can shuffle within each contributing bank. Bank blocks then stay together. A candidate still gets a different order from their neighbor. A themed section still reads as one, and a shared case still sits with its questions.
None of these touch impersonation. Nothing in the assembly layer does. For what the delivery layer can and cannot see, read can online exams detect cheating.
Are two randomized forms equivalent?
No — and this misconception is worth a paragraph.
A random draw produces different samples of items. Equated forms are something else. They are forms placed on a common score scale by a deliberate statistical procedure. Randomization is a sampling method. No amount of randomizing turns it into equating.
The difference shows up at appeal. Suppose your defense rests on "every candidate sat an equivalent paper". A random draw does not give you that. Two draws from the same bank can differ in average difficulty. That difference reaches real candidates as marks.
You can measure it afterward. Compare the forms on candidate count, mean, pass rate, spread, and average item difficulty. For each pair, compute the score gap, an effect size, and a significance test. StudyDrome Exam Manager runs Cohen's d and a Welch's t-test across published forms. It warns when a pair meets two conditions at once: the difference is significant, and the gap is wider than 3 points. Both conditions matter. In a large cohort, a tiny gap can be significant and still mean nothing.
So the honest sequence runs: randomize if the security case justifies it, measure equivalence, then be ready to say what you found. Equivalence is a finding. It is never an assumption.
Can your bank actually support random selection?
Random selection is a property of the bank, not of the exam. A thin bank makes randomization worse than a fixed form. It adds variance without adding coverage.
Four questions decide it:
- Depth per topic, not in total. A 600-item bank holding 400 items on one topic cannot sample a balanced paper.
- Spread across difficulty. If nearly every item sits in the middle band, every draw yields the same exam in different words.
- Quality, not just count. Items flagged as poor still occupy bank space. Do not count them as available.
- How much has never run. Items never administered have no statistics. A mostly untested bank is a bank you cannot predict.
StudyDrome Exam Manager reports this directly. It cross-tabs a bank by difficulty band and topic. It warns when a band holds fewer than five good questions. It warns again when more than a fifth of the bank has never been administered.
Note the boundary, because it is easy to misread. That report tells you whether a balanced paper can be built. It does not build one. No constraint solver picks items against a target blueprint. You still assemble the paper. The report only tells you whether the attempt is realistic.
A checklist for choosing
- Write down which risk you are managing: copying from a neighbor, a shared answer key, or a leaked bank. Each maps to a different switch.
- Ask whether you are standard setting on this paper. If an item-by-item method is in play, use a fixed form.
- Divide your cohort by your bank size. If items would collect only a handful of responses each, your statistics will not survive that division.
- Check depth per topic and per difficulty band, never the bank total.
- Count how much of the bank has never been administered before you rely on it.
- If you randomize, decide in advance how you will demonstrate equivalence, and to whom.
- Prefer shuffling within banks when your paper has themed sections or shared case material.
- Run a form comparison after the sitting. Record the result whether or not anyone asks.
Frequently asked questions
Should you randomize the order of questions in an exam?
Randomize the order when candidates sit together, or sit at the same time. It defeats copying from a neighboring screen. Leave it alone when the paper has a deliberate sequence, such as a case that builds across several questions. A middle option exists on many systems. Shuffle within each contributing bank so that blocks of related questions stay together while the order still varies across candidates.
Does randomizing exam questions stop cheating?
It removes specific opportunities rather than the problem. Order randomization prevents screen copying and messages about a question number. Drawing different items per candidate devalues a leaked paper. Neither addresses a second device, a person in the room, or someone sitting the exam on another's behalf. Randomization is one layer of an integrity plan, never a substitute for one.
Are randomized exam forms comparable to each other?
Not automatically. A random draw produces different samples of items. It does not place those forms on a common score scale. Two draws from the same bank can differ in average difficulty, and that difference is reflected in the marks candidates receive. You establish comparability afterward by comparing the forms on mean, pass rate, spread, effect size, and a significance test. The randomness of the draw establishes nothing.
How big does a question bank need to be for random selection?
No single number works, because depth per topic matters far more than the total. A bank needs enough good items in every topic and every difficulty band you intend to sample. Two further checks matter. How many items carry a poor-quality flag, and how many have never been administered? A large bank that is mostly untested cannot support a predictable draw.
What is LOFT, and do you need it?
Linear-on-the-fly testing produces a unique paper for each candidate against a statistical target. Every form matches a specification rather than merely being random. Cirrus lists it on its security page (checked 7 August 2026). Most institutional exams do not need it. A well-stocked bank with per-bank quotas achieves topic balance, and a subsequent form comparison demonstrates equivalence. LOFT matters when a regulator demands per-candidate forms built to a fixed statistical profile.
Should answer options be randomized too?
Usually yes, and it is a separate setting from question order. Shuffling the options defeats a shared answer key, the cheapest cheating method there is. Two exceptions apply. Options with an inherent order, such as numeric ranges or a date sequence. And any option like "none of the above", which has to stay in place. Those items should keep their authored order.
Where to go next
If you are setting this up for the first time, start from the bank rather than the exam. Assembly options only matter once there is something worth sampling. The fastest route to a defensible paper is better items, not a cleverer draw. Writing flaw-free items is where that begins, and the complete guide to item analysis covers how to read the results afterward.
If you already randomize and have never compared your forms, do that before the next cycle. It is a short piece of analysis. It either confirms what you assumed, or tells you something you needed to know. For what happens to the paper the moment it goes live, see what freezes at publish.