WHEN SHOULD I USE OPEN-ENDED
QUESTIONS?
University
of Wisconsin Cooperative
Extension
Wisconsin Tobacco Control Board Monitoring
and Evaluation Program
This document was produced through a grant from the Wisconsin
Tobacco Control Board, dated April 2002
This tip sheet offers some suggestions about how to choose open-ended
vs. closed questions when constructing a tool to collect evaluation
information. Open-ended questions are questions that do
not provide response options. They ask the respondent to answer
in his or her own words. "Closed," or "forced-choice"
questions (also known as "multiple-choice" questions) are
questions that offer a finite number of responses from which the
respondent can choose the best answer or more than one answer.
These two types of questions are used for distinct purposes and
require the evaluator to analyze and interpret data in distinct
ways.
You want to conduct a survey or series of interviews. You have
identified the evaluation questions you want to answer. As you
choose or design your own questions to collect the information,
you ask yourself what type of questions you should use: "Open-ended"
or "multiple-choice"? The following questions will guide
your choices:
-
How will you use the information from each question?
-
How will you use information from distinct questions to understand
more about subgroups of your sample population?
-
How do you plan to analyze the data?
- Given your answers to 1-3 above, what are the advantages
and disadvantages of using open-ended vs. "closed,"
or forced-choice questions?
How will you use the information?
- To understand attitudes among members of the county's voting
population? In this case, a "multiple-choice" or closed question
may be the best option. (See examples at right.)
- Are you asking a question because you would like to understand
why people choose to support an issue? In this case,
an open-ended question may or may not serve your purpose. It
is a good rule of thumb to use open-ended questions sparingly
on a written survey.
- You may choose to mix closed and open-ended questions to fulfill
both goals.
- It is also helpful during interviews, both face to face and
over the telephone, to offer some multiple-choice questions.
This question forces a response:
Would you support a law that prohibited smoking in workplaces
within the City of Richmond?
- Definitely yes
- Probably yes
- Probably no
- Definitely no
- Not sure
This question asks for more depth:
Please tell us why you answered the previous question the way
you did.
What demographic questions will allow you to understand differences
among subgroups of your sample population?
- If you survey the general population of your county about
attitudes to a possible ordinance to restrict smoking in workplaces,
you will most likely want to use demographic questions so that
you can analyze levels of support according to age, smoking
status, and possibly education level. Clarify in advance what
you really need to know and how the questions will produce the
data.
How do you plan to analyze the data?
- Open-ended questions, or questions that do not offer response
choices, are designed to collect narrative responses.
Narrative responses can be more complex to analyze and report.
They most often require more interpretation than well-designed
forced-choice questions.
The table below offers some advantages and disadvantages of using
open-ended and "closed," or "forced-choice" questions. Two examples
of open-ended and closed questions illustrate the potential for
very different types of responses.
Advantages and disadvantages of using
open-ended vs. "closed," or "forced-choice" questions
|
|
Open-ended questions
|
"Closed," or "forced-choice" questions
|
|
Example 1:
|
How has smoking affected your health?
|
Please indicate whether you strongly disagree, disagree,
agree, or strongly agree with the following statement:
Smoking has negatively affected my health.
- Strongly disagree
- Disagree
- Agree
- Strongly agree
- Don't know
|
|
Example 2:
|
What type of assistance do you think would most help you
quit smoking?
|
What type of assistance do you think would most help you
quit smoking? (Please choose only one option)
- Medication
- Counseling
- Combination of medication and counseling
- More support from family and friends
- None of the above
|
|
Advantages
|
- Allow respondents to answer in their own words.
- Answers aren't forced.
- Helpful to explore things for which you don't yet have
a hypothesis or theory.
- Provide more "richness" or "depth" in your data (e.g.,
may help you explore "why" in more detail).
- Can help you identify possible responses options for
further quantitative research.
|
Easier to code.
Allow for statistical summaries of large number of cases.
If question is well-constructed,
can provide more clear-cut categories to measure knowledge,
skill, attitude, or behavior.
Reporting results may be more straightforward.
|
|
Disadvantages
|
- Analysis may require more time.
- More complex to code.
- More difficult to make clear-cut comparisons between
responses.
- Respondents with strong positive or negative opinions
may choose to take the time to answer. Those with neutral
but important observations may not answer.
- Stronger role of interpretation in analyzing data.
|
Risk influencing responses by forcing choices.
Order of options can affect results.
Possible response options may be omitted.
"Other" or "none of the above" response options are not
always informative
|
Return
to Good Data from Bad Questions
|
