header
Cooperative Extension Skip Navigation UW Extension
Local Program Evaluation in Tobacco Control
Home Sitemap Contact Search
Navigation

RESOURCES

About Our Program
Evaluation Manual
Multi Year Action Planning
Existing Data
Evaluation Planning

Evaluation Methods

Analyzing Data
Using Results
Resources
Restaurant and Worksite Surveys
Clean Indoor Air
Coalition Development
Youth Prevention
Upcoming Training
Conferences & Presentations

Download a copy of the free Adobe Acrobat Reader to view and print information provided as PDF files.
Get Adobe Acrobat Reader

WHEN SHOULD I USE OPEN-ENDED QUESTIONS?

University of Wisconsin Cooperative Extension
Wisconsin Tobacco Control Board Monitoring and Evaluation Program
This document was produced through a grant from the Wisconsin Tobacco Control Board, dated April 2002


This tip sheet offers some suggestions about how to choose open-ended vs. closed questions when constructing a tool to collect evaluation information. Open-ended questions are questions that do not provide response options. They ask the respondent to answer in his or her own words. "Closed," or "forced-choice" questions (also known as "multiple-choice" questions) are questions that offer a finite number of responses from which the respondent can choose the best answer or more than one answer. These two types of questions are used for distinct purposes and require the evaluator to analyze and interpret data in distinct ways.


You want to conduct a survey or series of interviews. You have identified the evaluation questions you want to answer. As you choose or design your own questions to collect the information, you ask yourself what type of questions you should use: "Open-ended" or "multiple-choice"?  The following questions will guide your choices:

  1. How will you use the information from each question?

  2. How will you use information from distinct questions to understand more about subgroups of your sample population?

  3. How do you plan to analyze the data?

  4. Given your answers to 1-3 above, what are the advantages and disadvantages of using open-ended vs. "closed," or forced-choice questions?

How will you use the information?

  • To understand attitudes among members of the county's voting population? In this case, a "multiple-choice" or closed question may be the best option. (See examples at right.)
  • Are you asking a question because you would like to understand why people choose to support an issue? In this case, an open-ended question may or may not serve your purpose. It is a good rule of thumb to use open-ended questions sparingly on a written survey.
  • You may choose to mix closed and open-ended questions to fulfill both goals.
  • It is also helpful during interviews, both face to face and over the telephone, to offer some multiple-choice questions.

This question forces a response:

Would you support a law that prohibited smoking in workplaces within the City of Richmond?

    • Definitely yes
    • Probably yes
    • Probably no
    • Definitely no
    • Not sure

This question asks for more depth:

Please tell us why you answered the previous question the way you did.

What demographic questions will allow you to understand differences among subgroups of your sample population?

  • If you survey the general population of your county about attitudes to a possible ordinance to restrict smoking in workplaces, you will most likely want to use demographic questions so that you can analyze levels of support according to age, smoking status, and possibly education level. Clarify in advance what you really need to know and how the questions will produce the data.

How do you plan to analyze the data?

  • Open-ended questions, or questions that do not offer response choices, are designed to collect narrative responses.

Narrative responses can be more complex to analyze and report. They most often require more interpretation than well-designed forced-choice questions.

The table below offers some advantages and disadvantages of using open-ended and "closed," or "forced-choice" questions. Two examples of open-ended and closed questions illustrate the potential for very different types of responses.


Advantages and disadvantages of using open-ended vs. "closed," or "forced-choice" questions

 

Open-ended questions

"Closed," or "forced-choice" questions

Example 1:

How has smoking affected your health?

Please indicate whether you strongly disagree, disagree, agree, or strongly agree with the following statement:

Smoking has negatively affected my health.

  • Strongly disagree
  • Disagree
  • Agree
  • Strongly agree
  • Don't know

Example 2:

What type of assistance do you think would most help you quit smoking?

What type of assistance do you think would most help you quit smoking? (Please choose only one option)

  • Medication
  • Counseling
  • Combination of medication and counseling
  • More support from family and friends
  • None of the above

Advantages

  • Allow respondents to answer in their own words.
  • Answers aren't forced.
  • Helpful to explore things for which you don't yet have a hypothesis or theory.
  • Provide more "richness" or "depth" in your data (e.g., may help you explore "why" in more detail).
  • Can help you identify possible responses options for further quantitative research.
  • Easier to code.
  • Allow for statistical summaries of large number of cases.
  • If question is well-constructed, can provide more clear-cut categories to measure knowledge, skill, attitude, or behavior.
  • Reporting results may be more straightforward.
  • Disadvantages

    • Analysis may require more time.
    • More complex to code.
    • More difficult to make clear-cut comparisons between responses.
    • Respondents with strong positive or negative opinions may choose to take the time to answer. Those with neutral but important observations may not answer.
    • Stronger role of interpretation in analyzing data.
  • Risk influencing responses by forcing choices.
  • Order of options can affect results.
  • Possible response options may be omitted.
  • "Other" or "none of the above" response options are not always informative
  • Return to Good Data from Bad Questions