What are the most common MCQ item-writing flaws?
The most common MCQ flaws include implausible distractors, negatively worded stems, "all of the above" and "none of the above" options, longest-answer cues, grammatical cues between stem and options, and unfocused stems. Tarrant et al. (2006) found roughly 46% of nursing exam items contained at least one such flaw.
Item-writing flaws are violations of evidence-based question-writing rules. They matter because they distort measurement in one of two directions: some flaws make items easier for testwise students who don't know the content, while others make items harder for students who do. Either way, scores become less trustworthy — and Downing (2005) showed that flawed items can change pass-fail outcomes for borderline examinees on medical school exams.
Tarrant, Knierim, Hayes and Ware (2006) reviewed items from nursing examinations and found that roughly 46% contained at least one writing flaw. That figure is consistent with the broader pattern in health-professions education: locally written items, produced by faculty without formal item-writing training, are flawed at high rates.
How many exam items carry a writing flaw?
Items with at least one writing flaw
The flaws that appear most often
Drawing on the taxonomy of Haladyna, Downing and Rodriguez (2002) and Case and Swanson's NBME item-writing guide, the flaws reviewers find most frequently are:
Flaw | Why it's a problem |
|---|---|
Implausible distractors | Options no one selects shrink the effective number of choices and inflate guessing odds |
Negatively worded stems | "Which is NOT…" items are easily misread and test reading care more than knowledge |
"All of the above" / "none of the above" | Reward partial knowledge and elimination strategies rather than mastery |
Longest option is correct | Writers pad the keyed answer with qualifiers; testwise students pick the longest option |
Grammatical cues | The stem's grammar (e.g., "an…") fits only the correct option |
Unfocused stems | The stem asks no clear question, forcing students to read all options to infer the task |
Word repeats (clang associations) | A word from the stem recurs in the correct option and flags it |
How to catch them
Most flaws are visible on inspection, so a short peer-review pass with a checklist before an exam is administered catches the majority. Two focused checks are especially high-yield: the cover-the-options rule exposes unfocused stems, and scanning for negative wording and "all of the above" options takes seconds per item.
The flaws that survive review — chiefly implausible distractors — show up in the data instead. Post-exam item analysis reveals distractors that attracted almost no responses, which is the statistical signature of an option that isn't doing its job. Platforms that compute item statistics automatically, such as StudyDrome, make this a routine part of every exam cycle rather than a special project.
For a full walkthrough of the rules with before-and-after examples, see how to write flaw-free MCQs.