A Practical Guide to Writing Quizzes
Many quizzes test whether someone can recall a fact. Few check to see if they really understand it. In corporate training and education alike, this is an ongoing problem; Assessment sounds good on paper but fails to show whether students can apply what they have learned. The result is increased pass rates and a false sense of confidence in the effectiveness of the training program.
The good news is that writing better questionnaires doesn’t require a degree in psychometrics. It requires being more intentional about what each question is designed to measure and avoiding a few common pitfalls that make questions more predictable than truthful.
Creating Quizzes That Measure Understanding
Start with What You Want to Measure, Not What You Want to Ask
Before writing a single question, define the learning objective it outlines. This sounds obvious, but in reality, most quiz writers start with the content (“We talked about phishing in module 3,”) and then write the question about the details (“What percentage of breaches involve phishing?”). Those tests are memorizing, not understanding.
A better way is to ask yourself: What should the student be able to do after completing this module? If the objective is “Identify suspicious emails,” then the question should present a situation—an email with hidden red flags—and ask the reader to examine it. The question tests ability, not math.
Avoid Questions That Can’t Be Answered Without Reading
Some question formats are so familiar that students develop correct guessing techniques without knowing the text. Notice these patterns:
- “All of the above” as the correct answer.
Students quickly realize that if “all of the above” comes up, it’s usually okay. If you must use it, be sure that it is sometimes wrong. - One answer is obviously longer or more detailed than the others.
Quiz writers often put more effort into calling out the right answer, which makes it stand out. Keep all options roughly the same length and level of detail. - Badly worded questions like “Which of the following is NOT…”
These test attention more than knowledge. If you need to check if something is missing, rephrase it as a positive question with a status. - True/false questions on topics with different names.
Binary options give readers a 50% chance of guessing correctly. Keep True/False for clear facts, and use multiple choice or short answer for anything that requires judgment.
Write Audible Errors
Wrong answers to a multiple choice question are just as important as right. If the bugs are clearly wrong, the question becomes a little easier no matter how good the title is.
Effective misconceptions come from genuine misconceptions. If you’ve been training people on a topic for a while, you already know the common mistakes—those are your pitfalls. For example, in the data privacy question, the question “When would you share customer data with a third party?” it should include options that reflect a genuine understanding, such as “When the customer made the purchase” or “When the third party signs a standard NDA.” This makes sense because it shows real thinking errors.
If you don’t know the common misconceptions, answer the questions once and see what wrong answers people tend to choose. That data tells you where the confusion resides.
Use Scenarios and application questions
The easiest way to move from remembering to understanding is to put the student in context. Instead of asking “What is the first step in the incident response process?”, describe the specific incident and ask what should happen next. The student must see the situation, remember the process, and apply it,
Situational questions take longer to write, but they give you more useful data about student readiness. A student who can repeat the five steps of responding to a situation but cannot identify which step applies in a particular situation has not really learned the story.
Add Explanations to all answers
One of the most used features in query creation is the description field. If a student gets an answer wrong, showing them the correct answer alone does not help them understand why they were wrong. A brief explanation that corrects the misconception behind each bug turns testing into a learning moment.
This is especially important in self-paced training where there is no instructor to provide context. Even with correct answers, a short explanation emphasizes the idea. “Okay. You’re going to move up to the security team because the indications suggest a violation of the law, not just a violation of policy.” is more important than the green marker.
Keep the Language Simple and Precise
Vague words are the enemy of good testing. If the student gets the question wrong because they interpreted it differently than you intended, the question failed—not the student.
Avoid double negatives, vague adjectives like “sometimes” or “usually,” and jargon that wasn’t clearly taught in training. Each question must have exactly one defensible answer. If you find yourself writing lengthy reasons why one option is “better” than another, the question needs to be rewritten.
Ask someone off-topic to review questions before publishing them. New eyes capture the ambiguities the author cannot see.
Measure, Then Improve
Writing good questions is repetitive. After your questions have been taken by a reasonable number of students, review the data. Questions where almost everyone gets the answer right aren’t necessarily right—they might be too easy. Questions with an even distribution of options across the options may be less worded than really difficult.
Look at the discrimination index: do high-achieving students get this question more often than low-achieving students? If not, the question isn’t measuring what you think it is.
The goal is not to make the questions difficult. It’s about getting them more honest about what students really know, so you can focus your training effort where it matters most.
