Choose One: Is Multiple Choice Losing Its ___?
When it comes to test questions, the multiple-choice format reigns, and for good reason. The competition is weak.
Consider the true/false format. The student has a 50/50 chance of guessing the right answer. Make it more reliable by asking the learner to correct the false statements, and you get the student who reiterates the statement with a "not" thrown in. Finally, you disallow that lame trick and you’re still left with a problem. I can’t recall the citation, but I remember reading that false statements on exams reinforce false information.
Matching also has a probability problem. If you build a matching exercise that contains a relatively small number of items, students can use the process of elimination to improve their odds of guessing correctly. It’s more reliable and probably just as easy to create separate multiple-choice questions.
Having eliminated true/false and matching, short answer and the essay are the only competition.
Even as early at 1971 Thorndike stated, "One of the ‘sacred cows’ of educational measurement is the multiple-choice item format."* He went on to discuss how optical scoring technology perpetuated the over use of the multiple-choice format. But, if you compare the use of multiple-choice with optical scanning to essay or short answer, it’s easy to see the appeal.
The short answer/essay type question is relatively easy to write, but grading a different story. To ensure reliability and objectivity in grading, you should create an answer key before reading a single student paper. Another good habit is to grade the first essay question on every student paper then move on to the second essay, etc. That’s a lot of time already spent on grading and we have yet to generate any statistics on our test questions.
Multiple-choice questions take quite a bit longer to write, but everything about the grading, including the statistics, is automated. Falling for this appeal is not the sign of a bad teacher -- quite the contrary. A good teacher strives for a valid and reliable examination. A good teacher also strives for immediate feedback, which is known to improve student performance.
This June, the College Board announced changes to the SAT I: Reasoning Test. One of the changes is the addition of an essay to evaluate the student’s writing skill. The College Board intends to have two readers grade each essay to assure inter-rater reliability. This is the method that they have used on the SAT II: Subject Exams with good results.
I hate to sound like a cynic, but this year the College Board was 12 days late grading an additional 20,000 unexpected SAT II essays. When the College Board implements the new SAT I, they will be grading an additional 2 million essays (based on 2000/2001 stats). Seems like a pretty daunting task to me. **
- None of the above
*Thorndike, Robert L., ed. Educational Measurement. 2nd ed. Washington, D. C.: American Council on Education, 1971.
**The New SAT