How to Write Multiple-Choice Questions for Workforce Training

March 16, 2016

Resource: Blog Industry: Industrial

Tags: Vector LMS and Training Management

A lot of you write test questions for online training. Or even paper-based tests you’re still delivering the old-fashioned way (good on you!).

Maybe you’re doing it with an eLearning authoring tool, such as the ones from Articulate, Adobe, or Lectora. Or maybe the learning management system (LMS) you use at work has a built in tool for creating online quizzes.

No matter how you’re writing tests for training, you may sometimes find yourself wondering about the best practices for writing standard question types. (By the way, instructional designers often use the wonky phrases “assessment” for a test and “assessment item” for a question within a test.)

We’ve got a few of those best practices for you below. Hope this helps you with your question writin’. 🙂

General Tips for Writing Test Questions

Let’s cover some general tips for writing test questions (“assessment items”) before we zero in on multiple-choice questions.

We’re going to separate these into Do’s and Don’ts.

What To Do When You Write Questions: General Tips

Here are some things that you SHOULD DO when you’re writing questions for tests.

Do Create Questions for All of Your Learning Objectives

Make sure your questions assess learning objectives and nothing but learning objectives.

And make sure your questions cover all of the learning objectives (it can be easy to miss one or more sometimes).

Having a checklist that you refer to later may help you ensure you’ve properly assessed each learning objective.

You can write more than one question per learning objective.

Don’t know what a learning objective is? Here’s a good guide to learning objectives.

Do Create Questions that are True Assessments of Your Learning Objectives

In workforce training, most training is intended to teach people to do something. As in, perform a task or skill.

And, in most cases, your goal for creating the training isn’t to have the employee later (a) demonstrate knowledge on the job or (b) correctly answer a multiple-choice question.

So think hard before you sit down to write a multiple-choice question. Is this really assessing the workers’ ability to satisfy the learning objective, or are you just working quickly in default mode and writing a multiple-choice question when a real skills performance might be more appropriate?

I’m not saying there’s never a time to use multiple-choice questions. But I am saying they tend to be overused, and there may be better options.

Read these articles on reliability and validity in testing and on fidelity in testing for more on this.

Do Write Questions that Anticipate Common Misunderstandings/Errors

Now, this is different than a trick question. And it’s OK and even good to write questions like this.

These kind of questions can help you identify people who only “kind of” understand, or understand superficially, or have a common misunderstanding that needs to be corrected.

Remember this is valuable because testing is just another opportunity to teach and provide feedback.

Do Stick to One Point per Question

Write questions that address one point per question.

Avoid writing questions that attempt to assess multiple points.

Do Include Feedback

Find a way to include feedback to inform the employee if they got the answer correct or incorrect, and what the correct answer is, or why their answer is wrong, or how they can come to the correct answer.

This issue of feedback is key: feedback has been shown to have a significant influence on learning.

This can be easier to do with online systems, and is a nice benefit of the kind of automation you can get with online learning management systems, but you can do it after a test is turned in if you’re grading by hand.

Do Review and Edit Questions

Don’t just write your questions and let ’em out in the wild. Review all of your tests and test items carefully before you deliver them to employees.

During your review, check:

• for poorly written questions that are confusing or vague; check for
• for typos, spelling and grammar errors, and any other miscues you don’t want to be associated with; check
• the reading level to see if it’s appropriate for your testing population;
• to make sure the questions don’t include a bias, putting one sub-group of employees at an unfair advantage or disadvantage; check for
• for material that’s offensive, insulting, or needlessly controversial or disturbing

If you can, have another person (or even better, a team) review them as well–it’s always hard to review your own work. Then maybe see if you can get a small group of employees to “beta test” your test as a pilot.

What NOT To Do When You Write Questions: General Tips

Now let’s look at some things you SHOULDN’T DO when writing questions for tests.

Don’t Write Trick Questions

Don’t write trick questions.

You’re creating assessments and writing assessment items (“questions”) because you want to see if employees know something. In particular, if they know how to do something they need to do on the job.

You’re not writing a test item to see if employees are especially skillful at taking tests or at recognizing trick questions.

So, if your goal is to determine if employees know something, and not to see how clever they are at sniffing out trick questions, don’t write trick questions. No matter how tempting it may be.

Don’t Focus on Questions that Require Recognition

You can write a multiple-choice question (or other type of question) so that all the employee has to do is recognize the correct answer (because it’s written right there in the answer options).

While this type of recognition may help assess learning and even support learning to an extent, tests that require the worker to actively recall (from “scratch,” if you will) or apply the information are even better.

Don’t Write Questions that Test Only Recall or Recognition

While recall has its benefits (read up on the testing effect for more on that), application is even better.

Avoid writing questions that focus only or too much on recall/recognition. Try to include questions that require application.

(For more context on recall/recognition, application, and other forms of learning, see this article on Bloom’s taxonomy and/or the part of our interview with Dr. Will Thalheimer when he talks about retrieval, recall, and recognition).

Don’t Help Employees Identify Correct Answer Based on Tips from Grammar

You can sometimes accidentally make it easier for an employee to answer a question correctly just with the grammar you use.

For example, if the question is phrased in a singular manner, and all answer options are plural except one answer option that’s singular, that singular answer option is probably the correct answer.

Or, for another example, if your question stem ends with “an” and the four answer options begin with “cat,” “dog,” “leopard,” and “otter,” employees will know that the correct answer is “an otter.”

Review your questions to make sure you’re not tipping your hand and making it easy for employees who don’t know the correct answer to use grammar tips to identify the correct answer.

Don’t Write Questions That Provide Answers to Other Questions

Try to avoid writing a question that includes information that makes it easier for employees to answer a second question.

Don’t Write a Question Because It’s “Easy” to Write or Grade

If you want to create effective assessment items (“questions”), then do it. Don’t take the easy way out and create an assessment that doesn’t do what you want it to do.

We all get tired sometime; I get it. It can be tempting to create a question in a certain format just because it’s easier to write. Or, thinking down the road, because it’s easier to evaluate/grade.

But don’t do that; don’t give in to your weaker moments. Be a training superstar and write the best question you can–one that truly assesses whether or not employees understand the content.

Don’t Write Questions That Are Phrased with Double Negatives–and Try to Avoid Negative Phrasings Too

Avoid writing questions that are phrased with double negatives. For example, a question that asks “Which of the following is NOT unnecessary?” has a double negative (“not” and “unnecessary”) and can be confusing. Rewrite a question like this actively, to something like “Which of the following is necessary?”

You should always avoid double-negatives, as we just said. But in addition, you may want to be careful about questions that include negatives of any sort, and see if you can rewrite it in a more active manner. For example, “Which of the following tools should not be in a hot work area?” could be rewritten actively as “Which of the following tools is forbidden in a hot work area?”

Tips for Writing Multiple-Choice Questions

Now that we’ve given some general tips on test and question writing, let’s give some tips that are specific to multiple-choice questions.

Parts of a Multiple-Choice Question

Let’s begin by talking about and naming the different parts of a multiple-choice question. We’ve written a sample multiple-choice question below and have labeled its different parts.

As shown above, multiple-choice questions have four different parts:

• Stem: The base of question (in this case, “What color is the sky?”).
• Answer options: The possible answer choices. This includes the correct and incorrect answers. In this example, it’s A, B, C, and D (or red, orange, blue, and purple).
• Correct answer: The answer option that is correct. In this example, it’s C, or blue.
• Distractors: All of the answer options other than the correct answer. In this example, it’s A, B, and D, or red, orange, and purple.

Now that we’ve got our terms down, let’s look at some guidelines for creating each part of a multiple-choice question:

Tips for Writing the Stem of a Multiple-Choice Question

Here are some tips for writing the stem:

• Keep the stem short and simple.
• Keep the wording precise. Don’t write something that’s vague, ambiguous, or confusing.
• Try to write the stem as a complete question (such as “What color is the sky?”) instead of fill-in-the-blank (“The sky is _____.”) when possible.
• If you do write a fill-in-the-blank-style question, put the “blank” space at the end of the stem and not in the middle when possible. For example, write this (“The author who wrote Hamlet is ________.”) instead of this (“The writer _____ wrote Hamlet.”).
• Put any wording that’s repeated in every answer alternative into the stem, so it only has to be written once.

Tips for Writing the Answer Options of a Multiple-Choice Question

Here are some tips for writing the answer options:

• All answer options should be “plausible.” This means they are all reasonable alternatives. Don’t write answer options that obviously can’t be correct.
• Write all answer options in a simple, clear manner.
• Aim for at least three answer options per question.
• There’s no single “correct” number of answer options to include in a question. Four or five seems to be a standard.
• It is not absolutely necessary to always have the same number of answer options for each question. Try to remain consistent when possible, though. If you need to add one or more answer options beyond your standard, that’s OK if there’s a good reason but don’t do it without a good reason. If you can’t seem to come up with enough answer options, think hard and try to create one or more plausible one. But if you can’t find enough plausible answer options, it’s better to go with fewer answer options than to create non-plausible answer options.
• Avoid writing answer options like “B and C”.
• Avoid using “All of the above” and “None of the above” as answer options. “All of the above” tends to be the correct answer a large number of times when it’s used, and so it helps employees guess the correct answer. Plus, if an employee knows that just one of the answer options is incorrect, the employee will know that “All of the above” is incorrect, too. And if “None of the above” is the correct answer, the question item doesn’t tell you if the employee knows the correct answer.
• Use the words the words all, always, and never with caution. These words tend to indicate an incorrect answer.
• Be careful when using words like usually, generally, or typically. These words tend to indicate a correct answer.
• Try to write all answer options in a similar format. If you know your grammar a bit, this includes writing your answer options in a parallel structure. If you’re not familiar with “parallel structure,” that link you just passed should help.
• Try to keep all answer options to a similar length.
• Be especially careful not to write the correct answer in a way that’s different than the other answer options (for example, notably longer–this is common).
• When possible, put the answer options in a logical order (alphabetic, numeric, some other logical order) or randomize the order of the answer options.
• Avoid answer options that overlap.
• Avoid writing more than one answer option that is arguably correct (unless it’s a multiple-response question that accepts multiple answers).

Tips for Writing the Correct Answer of a Multiple-Choice Question

Here are some tips for writing the correct answer:

• Don’t copy the correct answer word-for-word directly from the training materials. Instead, paraphrase the correct answer so it means the same thing in slightly different words.
• Double-check to make sure that the correct answer is written in a similar fashion as the other answer options. Correct answers are often written in a different manner (for example, they’re often longer) and as a result they can help employees guess the correct answer.
• “Mix up” the correct answers, so that they’re randomly distributed amongst A, B, C, and D not always/often the same value–such as C. For example, in a 10-question test, these correct answers are mixed up well (adcbbadbca) and these correct answers are not mixed up well (acbcdccbcc). Also, watch for obvious patterns (abcdabcdab). It may help you to know that question writers tend to make B and C the correct answer most often.
• Related to the last point (and to our earlier point about trick questions), don’t try to play “head games” with employees by doing things like having the correct answer be C for ten questions in a row. Again, you’re trying to create an effective assessment of workforce training (and learning), not a psychology test.

Tips for Writing the Distractors of a Multiple-Choice Question

Here are some tips for writing the distractors:

• Write distractors that could be correct (learning experts call this making the distractors plausible). Don’t write nonsense answer options. This can be hard, but dig deep and put in the work to come up with good distractors.
• Don’t write joke distractors. There are other times to be funny.
• Don’t include distractors that cover material that you can’t be confident all employees are familiar with as a result of the training. This can create a biased question that allow some employees to answer the question correctly more easily than other employees because of something they are familiar with from outside of the training.

Multiple-Choice Questions in Non-Traditional Formats

Remember that a lot of eLearning authoring tools (Articulate, Captivate, etc.) allow you to create what amounts to a multiple-choice question in non-standard formats. For example, maybe the learner will click one of three doors, or select one of four packages.

Always remember you can get more creative with these, and doing so may make the question more authentic and/or more engaging.

Conclusion: Writing Multiple-Choice Questions for Tests

Well, that’s what we’ve got for you.

What are your own thoughts? About writing tests and test items in general? About writing multiple-choice questions?

If you found this interesting, you might enjoy these other articles related to testing as well:

• Testing Best Practices
• Writing multiple-choice questions
• Writing true/false, matching, dragging, and other types of questions
• Using scenario-based learning and assessments
• Testing and fidelity
• Testing, reliability, and validity
• The “testing effect” and the forgetting curve

Also, stay tuned for a future blog post on the topic of performance assessments–assessments that ask a worker to perform a real-world job task. That’s coming, we think.

And don’t forget to download the free Guide to Effective Manufacturing Training below while you’re here.

How to Write Learning Objectives

All the basics about writing learning objectives for training materials.

Download Free Guide