What is distractor analysis, and what is a non-functioning distractor?

Distractor analysis extends item analysis below the level of right/wrong. For each multiple-choice item it tabulates how many examinees chose every option — the key and each distractor — often split by ability group (for example, the top and bottom scoring thirds). The premise is simple: a distractor only contributes to measurement if some examinees actually find it plausible, and specifically if less-knowledgeable examinees find it more plausible than knowledgeable ones do.

What a functioning distractor looks like

A healthy option-response pattern shows:

• the key selected more often by high scorers than low scorers (a positive point-biserial);
• each distractor selected by a meaningful share of examinees, drawn disproportionately from the lower-scoring group;
• no distractor that attracts the top group more than the bottom group — that inverted pattern suggests ambiguity and often accompanies a negative point-biserial.

Non-functioning distractors

A non-functioning distractor (NFD) is usually defined by a selection-frequency threshold: an option chosen by fewer than 5% of examinees. The threshold is a convention rather than a named standard — it predates the study most often cited for it, and Tarrant, Ware and Mohammed record it as what the field already used.

That study's own definition has two parts joined by or: an option with a response frequency below 5%, or one with a positive discrimination — chosen more by strong examinees than weak ones. An option doing the wrong work is as useless as one doing none. Many analysis tools implement only the frequency half, so check which definition a report applies before comparing figures across systems.

NFDs are pervasive. Tarrant, Ware and Mohammed (2009) reviewed 1,542 distractors across 514 four-option items in an undergraduate nursing programme. Only 52.2% of distractors were functioning, and 10.2% were chosen by nobody at all. Just 13.8% of items had all three distractors working, while 12.3% had none. Items with more functioning distractors were more difficult and more discriminating. Their conclusion is now widely accepted in medical education: write "as many options as is feasible given the item content and the number of plausible distractors; in most cases this would be three." Writing guidance has followed — Haladyna, Downing and Rodriguez (2002) advise writing as many good distractors as the content supports rather than padding to a fixed count, and the NBME item-writing guidance emphasizes plausibility over number.

Distractor efficiency

Distractor efficiency rolls the option-level picture into one number per item: the share of that item's wrong options still doing work.

Distractor efficiency = (total distractors − non-functioning distractors) ÷ total distractors × 100

A four-option item has three distractors. If one of them is non-functioning, efficiency is (3 − 1) ÷ 3 × 100 = 66.7%. An item with no wrong options at all scores 100% by convention, because there is no inefficiency to measure. A true/false item sits at the other extreme: it has a single distractor, so if nobody picks it efficiency falls to 0% — which means the item was easy, not that its options were badly written. That is why true/false items are normally left out of bank-level non-functioning-distractor lists.

Using the analysis in practice

After each administration, review the option-level table for every flagged item:

The best distractors encode real misconceptions — the wrong diagnosis students actually reach, the calculation error they actually make — which is why distractor review is as much a content-expert task as a statistical one. Over successive administrations, distractor data tells item writers which wrong answers earn their place in the bank. For worked examples of reading option-response tables, see the complete guide to item analysis, and for the flaws that produce weak distractors in the first place, see the most common MCQ writing flaws.