A lot of you write test questions for online training. Or even paper-based tests you’re still delivering the old-fashioned way (good on you!).
Maybe you’re doing it with an eLearning authoring tool, such as the ones from Articulate, Adobe, or Lectora. Or maybe the learning management system (LMS) you use at work has a built in tool for creating online quizzes.
No matter how you’re writing tests for training, you may sometimes find yourself wondering about the best practices for writing standard question types. (By the way, instructional designers often use the wonky phrases “assessment” for a test and “assessment item” for a question within a test.)
We’ve got a few of those best practices for you below. Hope this helps you with your question writin’. 🙂
Let’s cover some general tips for writing test questions (“assessment items”) before we zero in on multiple-choice questions.
We’re going to separate these into Do’s and Don’ts.
Here are some things that you SHOULD DO when you’re writing questions for tests.
Make sure your questions assess learning objectives and nothing but learning objectives.
And make sure your questions cover all of the learning objectives (it can be easy to miss one or more sometimes).
Having a checklist that you refer to later may help you ensure you’ve properly assessed each learning objective.
You can write more than one question per learning objective.
Don’t know what a learning objective is? Here’s a good guide to learning objectives.
In workforce training, most training is intended to teach people to do something. As in, perform a task or skill.
And, in most cases, your goal for creating the training isn’t to have the employee later (a) demonstrate knowledge on the job or (b) correctly answer a multiple-choice question.
So think hard before you sit down to write a multiple-choice question. Is this really assessing the workers’ ability to satisfy the learning objective, or are you just working quickly in default mode and writing a multiple-choice question when a real skills performance might be more appropriate?
I’m not saying there’s never a time to use multiple-choice questions. But I am saying they tend to be overused, and there may be better options.
Now, this is different than a trick question. And it’s OK and even good to write questions like this.
These kind of questions can help you identify people who only “kind of” understand, or understand superficially, or have a common misunderstanding that needs to be corrected.
Remember this is valuable because testing is just another opportunity to teach and provide feedback.
Write questions that address one point per question.
Avoid writing questions that attempt to assess multiple points.
Find a way to include feedback to inform the employee if they got the answer correct or incorrect, and what the correct answer is, or why their answer is wrong, or how they can come to the correct answer.
This issue of feedback is key: feedback has been shown to have a significant influence on learning.
This can be easier to do with online systems, and is a nice benefit of the kind of automation you can get with online learning management systems, but you can do it after a test is turned in if you’re grading by hand.
Don’t just write your questions and let ’em out in the wild. Review all of your tests and test items carefully before you deliver them to employees.
During your review, check:
If you can, have another person (or even better, a team) review them as well–it’s always hard to review your own work. Then maybe see if you can get a small group of employees to “beta test” your test as a pilot.
Now let’s look at some things you SHOULDN’T DO when writing questions for tests.
Don’t write trick questions.
You’re creating assessments and writing assessment items (“questions”) because you want to see if employees know something. In particular, if they know how to do something they need to do on the job.
You’re not writing a test item to see if employees are especially skillful at taking tests or at recognizing trick questions.
So, if your goal is to determine if employees know something, and not to see how clever they are at sniffing out trick questions, don’t write trick questions. No matter how tempting it may be.
You can write a multiple-choice question (or other type of question) so that all the employee has to do is recognize the correct answer (because it’s written right there in the answer options).
While this type of recognition may help assess learning and even support learning to an extent, tests that require the worker to actively recall (from “scratch,” if you will) or apply the information are even better.
While recall has its benefits (read up on the testing effect for more on that), application is even better.
Avoid writing questions that focus only or too much on recall/recognition. Try to include questions that require application.
(For more context on recall/recognition, application, and other forms of learning, see this article on Bloom’s taxonomy and/or the part of our interview with Dr. Will Thalheimer when he talks about retrieval, recall, and recognition).
You can sometimes accidentally make it easier for an employee to answer a question correctly just with the grammar you use.
For example, if the question is phrased in a singular manner, and all answer options are plural except one answer option that’s singular, that singular answer option is probably the correct answer.
Or, for another example, if your question stem ends with “an” and the four answer options begin with “cat,” “dog,” “leopard,” and “otter,” employees will know that the correct answer is “an otter.”
Review your questions to make sure you’re not tipping your hand and making it easy for employees who don’t know the correct answer to use grammar tips to identify the correct answer.
Try to avoid writing a question that includes information that makes it easier for employees to answer a second question.
If you want to create effective assessment items (“questions”), then do it. Don’t take the easy way out and create an assessment that doesn’t do what you want it to do.
We all get tired sometime; I get it. It can be tempting to create a question in a certain format just because it’s easier to write. Or, thinking down the road, because it’s easier to evaluate/grade.
But don’t do that; don’t give in to your weaker moments. Be a training superstar and write the best question you can–one that truly assesses whether or not employees understand the content.
Avoid writing questions that are phrased with double negatives. For example, a question that asks “Which of the following is NOT unnecessary?” has a double negative (“not” and “unnecessary”) and can be confusing. Rewrite a question like this actively, to something like “Which of the following is necessary?”
You should always avoid double-negatives, as we just said. But in addition, you may want to be careful about questions that include negatives of any sort, and see if you can rewrite it in a more active manner. For example, “Which of the following tools should not be in a hot work area?” could be rewritten actively as “Which of the following tools is forbidden in a hot work area?”
Now that we’ve given some general tips on test and question writing, let’s give some tips that are specific to multiple-choice questions.
Let’s begin by talking about and naming the different parts of a multiple-choice question. We’ve written a sample multiple-choice question below and have labeled its different parts.
Now that we’ve got our terms down, let’s look at some guidelines for creating each part of a multiple-choice question:
Here are some tips for writing the stem:
Here are some tips for writing the answer options:
Here are some tips for writing the correct answer:
Here are some tips for writing the distractors:
Remember that a lot of eLearning authoring tools (Articulate, Captivate, etc.) allow you to create what amounts to a multiple-choice question in non-standard formats. For example, maybe the learner will click one of three doors, or select one of four packages.
Always remember you can get more creative with these, and doing so may make the question more authentic and/or more engaging.
Well, that’s what we’ve got for you.
What are your own thoughts? About writing tests and test items in general? About writing multiple-choice questions?
If you found this interesting, you might enjoy these other articles related to testing as well:
Also, stay tuned for a future blog post on the topic of performance assessments–assessments that ask a worker to perform a real-world job task. That’s coming, we think.
And don’t forget to download the free Guide to Effective Manufacturing Training below while you’re here.
All the basics about writing learning objectives for training materials.