Designing Multiple Choice Questions -- Do's and Don'ts

From: http://web.uct.ac.za/projects/cbe/mcqman/mcqappb.html

General considerations
The task of designing good MCQs may be simplified by referring to past student work such as tutorials, class tests and examination answers, as these are rich sources of the type of errors and misconceptions which students frequently expose. MCQs designed with this material at hand will be relevant to the subject area and will be taken seriously by the students.

Test for significant learning outcomes.
The questions should be designed to test the learning objectives of the course, and not trivia associated with the subject matter. Questions should be recognized as being relevant to the goals of the course.

Test for the intended intellectual skills.
A question may unintentionally be a "test within a test". Look at the following stem:

"How many permutations are possible in a bridge hand?"

A student who knows how to calculate permutations and combinations (which is assumed to be the skill which is tested for in the context of an elementary Statistics course) will not be able to answer this question if he/she has never played cards, let alone bridge!

Pay special attention to the language used.
The level of the language should be within reach of the students, bearing in mind that there may be many in the class whose home language is not English. Use correct grammar throughout and avoid the use of jargon, unless you are specifically testing terminology. Remember that "second-language" students will take longer to read and understand a question. This, of course, is true for other types of examinations, in particular, those based on essay-type questions, where mis-reading the question by the student may lead to the writing of a good essay totally off the point envisaged by the examiner.

Be sensitive to cultural and gender issues.
Avoid turns of phrase and figures of speech that could reasonably be construed as racist or sexist, or which may have a cultural bias.

Avoid giving clues.
The correct answer may sometimes be deduced by "test-wise" students from unintentional clues linking the stem to the answer:

A group of islands is called an

a. peninsula
b. archipelago
c. moraine
d. polder

Here, there is a grammatical clue linking the stem to the correct answer b) in that the "an" at the end of the stem is the only article acceptable for the word "archipelago". Rather end the stem with "a/an".

Which one of the following publications is considered to be a prime source for research articles on
achievement testing?

a. Journal of Educational psychology
b. Journal of Educational Measurement
c. Journal of Clinical Psychology
d. Review of Educational Research

The correct answer d) is the only alternative to include the word "research", which is also found in the stem. In addition, it differs from the distractors in that it is not a "Journal", but a "Review".

Avoid using interrelated items.Questions should not overlap.


The question stem

State the problem clearly, simply and as concretely as possible. Avoid vague generalizations and do not include irrelevant information. It is essential that the students should know exactly what is expected of them.

The stem should contain most of the wording in order to reduce the reading load.

Avoid using negative stems. If this cannot be done, the negative words should always be highlighted by underlining or capitalisation:"Which one of the following is NOT an inert gas?"

The alternatives

They should be equally plausible and they should be internally consistent. The internal friction due to molecular cohesion in
fluids is called

a. surface tension
c. viscosity
b. fluidity
d. ductility
e. absorbance

Alternative d) has nothing to do with fluids and so can be eliminated by a student who is otherwise uninformed.

Only one of them should be unambiguously correct. Critical examination by a colleague is essential!

Try and ensure that they are approximately the same length.

Avoid "All of the above" (Analysis shows that these questions tend to be the easiest. This response shows that the question designer has run out of ideas!).

Use "None of the above" with care (Analysis shows that these questions are more difficult. On the plus side, they reduce the effect of guessing, but give no diagnostics on misconceptions).

The position of the answer should vary.