DEVELOPING A QUESTIONNAIRE
Writing a questionnaire is a creative task that comes more easily to some than to others. Faced with a blank screen or piece of paper, many people have no idea where or how to begin. This chapter provides some guidelines about putting together a good survey for beginners.
Like any writing assignment, drafts should be reviewed and comments provided on early versions of any survey. It is important that enough time be planned to allow for rewrites and learning how to write a survey good enough to distribute. One useful task is to have other students in the course be respondents for their classmates’ surveys.
They can “take” the survey and provide feedback before a final version is printed and distributed.
Sometimes it helps to get students to talk out loud about what they want to know.
Too often they have a very vague or broad idea about what to study. This makes it difficult to begin to write a clear and concise questionnaire. Ask them to carry on a conversation with a classmate about the topic. How would they begin? What kinds of things would they like to know that they don’t know already? Have them brainstorm in groups about behaviors, attitudes, and demographics related to the topic they wish to study.
As with all new skills, practice is important, so the more you can get students to review published questionnaires, poor quality ones in magazines and junk mail, and items printed in academic articles, the more they will see what goes into operationalizing, formatting, and designing a questionnaire.
Developing a Questionnaire
Class Discussion Questions
- What kinds of topics are most suited to questionnaires? Which ones are the least appropriate to surveys?
- What do they feel about on-line surveys versus hard copy ones? Which do they prefer and why?
- Ask students about the last time they were invited to complete a survey. What was it about, what kinds of questions were asked, how did they feel completing it?
- Ask who likes and who dislikes filling out forms. Then divide the class into two groups of likes and dislikes and have them specifically list the reasons. Finally, ask how they would do to make such forms or surveys more fun to
- Start collecting questionnaires, especially ones that are Distribute them to groups of students and have them “grade” the surveys and specifically discuss what is right and wrong about them. Have students look for surveys and invite them to bring the questionnaires to class for discussion in small groups.
- Distribute the methods sections of some academic articles and ask students to review the way variables were measured. Ask them to suggest other ways to measure the same ideas, or other ways of rewriting the same questions. This is good way to introduce the creative aspects of writing questionnaires, and to show how findings could be different if items were worded in alternative
Answers to Chapter Four Exercises
Many of these exercises have multiple correct answers and these are just some of the possible ones.
Interpret: What Do These Real Examples Tell Us?
1a. Good that it specifies a time frame (“during the past week”), but question implies that one was read (“which newspaper was that?”). Should prompt the interviewer to skip
the second part if respondent says “none.” A self-administered survey would have to indicate some branching either visually or in words.
1b. Like the previous one, some prompts should be given to the interviewer if respondent says “no” to seeing or hearing a show. “Political or social issues” might need to be defined with an example or two. A self-administered survey would also have to indicate some branching.
1c. Might be necessary to specify that you are asking about a political party, or who they voted for in the last election, or if they are actually registered with a party. But
it does say “generally speaking” so it should be clear that this is about an identity. On a self-administered survey, it should be made specific since there is no one to ask to clarify.
1d. Some might see this as a double-barreled item in that they could consider government and public affairs as two separate things. A self-administered questionnaire would use a Likert scale format for the choices (“most of the time” = 1, etc.).
2a. If living with both their own mother and father, the interviewer must skip to question 8 about what kind of work the father did, otherwise ask item A for the remaining answers to find out what happened to parents.
2b. closed-ended items are also provided in Item A, and for 8 E. Open-ended items are asked about occupation and place of work (Items for 8).
2c. Could be confusing, especially the box (IQ-3 Interviewer) with further instructions to check back on Q. 7.
2d. It is important that an interviewer rehearse this section given all the contingency items and branching. It also saves time during the interview and prevents the respondent from feeling that the interviewer doesn’t know what he or she is doing, and thus call into question the reliability and validity of the study.
Consult: What could be done?
- Such things as: bad format and biased instructions (assumes quality of service and food needs improving), items not numbered, more than one per line, vague questions without time frames (ate how often in the past week? Month? When?), items often have more than one “right” answer, options not mutually exclusive (cereal is a vegetarian meal and fruit can be a dessert; what if you are 5’ 5” tall?) or exhaustive (other items
Developing a Questionnaire
available in cafeteria), double-barreled items (ham and eggs), wordy biased items (“Some people feel…”), invoking an authority biases item (“a nutritionist from the Department of Health”), no branching (last item “Why do you thing she is not wrong” not only is poorly worded but assumes previous answer is “No”), no date when survey is due to be returned, invites comments but doesn’t say about what, scale from 1 to 10 doesn’t specify if 1 or 10 is the best quality food, and so on. Numerous mistakes to find.
- Many different ways of rewriting any or all of these
Test Items for Chapter Four
Answers for questions appear in bracketed capital letters.
- Compare the following:
- open-ended and closed-ended questions:
[open: allow the respondent to write out or talk about their responses using their own words, not choose from a selected list of answers
closed: standardized answers to select from, like a multiple-choice test item]
- rating and ranking:
[rate: to measure each response along a scale of intensity rank: to place a set of items in order of preference]
- mutually exclusive and exhaustive
[exclusive: not possible to select more than one response or value exhaustive: all possible values or responses are provided]
- anonymous and confidential
[anonymous: no information about the person completing a questionnaire is attached to the survey
confidential: information about the respondent is available but not revealed to anyone other than the researchers]
- Consider these items from a questionnaire you are asked to review. Describe the main problems with the wording of these items and how you can improve the
- How often in the past week have you listened to news on the radio?
- 1 – 2 times
- 3 – 5 times
- 5 – 7 times
[not mutually exclusive –what if you listened 5 times? – and categories don’t have equal intervals. make 1-3, 4-6, 7-9, etc. also, not clear if the question means how many days did you listen to the news on the radio. if it means “times” then it should have larger range of possible times: the last response should be “7 or more” for example]
- I enjoy watching sitcoms and drama shows on a regular
Strongly Agree Agree Disagree Strongly Disagree
[double-barreled: may like sitcoms but bnot dramas. avoid “and” and ask one or the other and make into two questions]
- Are you currently dating someone? Yes No
- How many times do you go out with that person in a typical week?
[no branching or contingency item: item 2 should only be answered if “yes” to item 1]
- Imagine you have a list of 4 recent movies and you want your respondents to evaluate them for you. Write a question that gets them to rate Then write a question that asks them to rank them. Which is better and why?
[rate: on a scale of 1 to 5 where 5 is best, rate each of the following four movies (for example, “harry potter” 4)
rank: take these four movies and rank them assigning a “1” to the movie you like best, a “2” to the one you feel is your second favorite, and so on. You can always rank them later on after you see the ratings for each movie, unless they are all tied and the respondent gives a “5” for each of them. asking someone to rank them gets you the order but it doesn’t tell you whether the #2 movie is liked or hated, just that it is not #1]