Writing Good Survey Questions
So, you've decided to do a survey, or maybe someone else has decided that you should do a survey. Before you start writing questions, you need to sit down and think.
- What are you trying to find out?
- What do you want to achieve by doing the survey?
- What will you do with the data when you've finished collecting it?
These questions will help you to define your aims
for the survey.
When you have worked out what you want your survey to achieve, it will be a lot easier to come up with questions to ask other people.
For example, suppose you want run a fund-raising event for a project. You'd like your event to be attended by as many people as possible, so you might survey your local community to find out things like:
- What sort of events do people enjoy attending?
- Are people willing to attend a fund raising event?
- How many people would be willing to attend a fund raising event?
- What time is the best time to hold such an event?
Designing the Questions
Step 1. Brainstorming
If you start out by thinking you have to write the most amazing questions ever, you'll probably never get anything done because nothing will seem to be good enough.
Instead, sit down and write as many questions as you can think of. Even better, sit down with a group of friends and brainstorm some questions. Don't worry about the quality of the
questions you write, we'll fix that up later. Just get as many down that you can think of.
For example, some initial questions for your fund raising event survey might include:
- Do you enjoy trivia nights?
- Do you enjoy going to the movies?
- Do you like going out to dinner?
- What sorts of events do you enjoy attending?
- When would you be able to attend a fund-raising event?
- How much would you be willing to pay for a ticket to a fund-raising event?
- Does it worry you that many children are homeless?
- Would you be willing to attend an event that is organised to raise money for homeless children?
- Do you have some spare money that you could donate to a charity for homeless children?
It's also a good idea to gather some
demographic information: this is information about the age group, gender, residence, occupation etc of the people you survey. Don't make these questions too
personal. They are useful because they let you know about the sort of people who have responded to your survey.
Types of Questions
There are two main types of questions:
- Closed Questions: the person chooses their answer from a list of provided options.
Open Ended Questions:the person gets to write their own response.
Closed questions give you quantitative data that is much easier to analyse later on, but they may prevent people from answering in the way that they wish, or may make people feel that there is no
applicable answer. Poorly designed closed questions can discourage people from completing your survey. However, open-ended questions can make answering a survey seem like too much hard work.
For example, if you wanted to ask the question "When would you be available to attend an event?", it is probably easiest to make it a closed question by providing a few dates as options, and a final option
of "None of these dates are suitable." If you make this question open-ended, you may find that everyone puts down a different date, and you will be no better off than before you did the survey.
If you wanted to ask, "What are your favourite games to play at a trivia night?", it might be best to leave it open-ended. It won't be so easy to tally up the results, but you might get some good ideas
of different games to play on your trivia night.
Look at each of the questions you are thinking of asking, and decide whether they would be best as open-ended or closed questions. Keep a balance of each type, and think about using some questions that are a
combination of both, by adding an "Other, please specify." option to your closed-ended questions, where it is appropriate.
Keep the Survey Short
Try to limit yourself to no more than \(20\) questions. If surveys are too long, people are less willing to answer them. Sometimes they might get bored and try to finish the survey as quickly
as possible by filling in any response at all, without reading the questions properly. Other people might refuse to respond to a survey if it's too long.
Try not to include too many open-ended questions requiring extended responses. This can also discourage people from completing your survey.
Think About the Order of the Questions
Think carefully about the order you arrange your questions in. You don't want to lead people to give the responses you want, or your data ends up being biased. There's a very funny example of
this on a "Yes, Prime Minister" episode. Why not check it out on the IMdb website?
For example, many people will say "yes" to attending a fund raising event if you ask the questions in this order:
- Do you realise that many children are homeless and starving in our city?
- Do you think we should do something to alleviate their suffering?
- Would you be willing to donate some money to help buy them some food and clothing?
They'd feel like a cad if they didn't.
However, if you ask the questions in this order:
- Do you find it difficult to find money to pay your bills?
- Do you think the government should do more to help homeless children?
- Would you be willing to donate some money to help by food and clothing for homeless children?
more people are likely to say "no" to the final question.
Try to arrange your questions so that they go from
- The least confronting to the most confronting.
- More general to more specific.
- Factual questions to questions seeking an opinion.
Try to ask questions that don't presume what a person's opinion is. Let people think for themselves. Use neutral rather than emotive language.
For example, "Does seeing homeless children break your heart?" is a bad question because it is too emotive, and virtually guarantees the answer "yes". Instead, you should ask something like
"How important do you think it is to help homeless children?"
- Not Important.
- A little important.
- Somewhat important.
- Very important.
You can also make statements and ask people if they agree or disagree with them. If you use this technique, it's a good idea to write these questions in pairs that state the opposite opinion to verify the responses. Sometimes people
will just tick the same response to a list of statements of this kind.
For example, if you write down the statement:
"The government does more than enough to deal with the problem of homelessness in our city."
you should pair it with
"The government does too little to help with the problem of homelessness in our city."
which expresses the opposite point of view. Then, provide options of
- Strongly Agree
- Strongly Disagree
for each statement. A person who replies "Strongly Agree" to both statements is obviously not taking the survey seriously.
Make sure all your questions are neutral.
Think About the Answer Choices for Closed Questions
Make sure that you include most of the most common answers. If you can't think of every possible response, it is always helpful to include "other" as an answer choice.
For example, if you ask "What is your favourite type of fund-raising activity?", you might provide the following answer choices:
- Trivia Nights
- Movie Nights
- Silent Auctions
It is helpful to run a small open ended survey to see what the common responses are. You might ask your friends or neighbours how they would respond to your
Some Response Types
- Multiple Choice: just provide a list of responses and ask your respondents (the people taking the survey) to choose one.
- Scales: give the respondents a scale that they can use to rate their feelings about a statement. For example,
- Ranking Responses: the respondent gets to rank each option. Keep the list of options short, or this becomes too difficult, and people start to just number the options in the order they appear.
Please rank the following fund-raising activities from 1 for your favourite through to 5 for your least favourite:
__________ Trivia Nights.
__________ Movie Nights.
__________ Sausage Sizzles.
__________ Cake Stalls.
- Numbers: sometimes you can just ask for a number as a response:
How much would you be willing to pay for a ticket to a trivia night organised to raise money for homeless children?
Look at each of your closed questions and decide on the best response type for each of them.
Collecting Your Data
Try to think how you will gather the answers to your questions before you start asking them. For example, are you going to interview people or are you going to ask them to complete the survey on a computer? If
you interview people, are you going to get someone to video the interviews so that you can write down the responses later?
It is important to make administering the survey easy, both for yourself and for your respondents.
The Survey Form
Your survey form (or
questionnaire) should be neat and easy to read and answer. Make sure it is easy to collect the data from it as well. Leave plenty of space for open-ended questions.
It's a good idea to type your questionnaire up using a word processor.
Think About How You Will Display Your Data
Look at each of your questions, and make sure that it is set out so that you can display your data in the way you want to in your report.
Will you use a
- bar graph
- pie chart
- line graph
- verbal explanation?
Verbal explanations work best for open-ended questions, tables are good for questions with simple yes or no answers, bar charts and pie charts work well if you want to show the proportion of each response, and line
graphs are good when you want to show how something changes.
For example, if you decide to have five options for the question, "What is your favourite fund-raising activity?", a bar chart or pie chart will be the best way to display your data.
Run a Pilot Survey
It is a good idea to test your survey out on a small group of people like your friends and neighbours before using it on a bigger group. This is called running a
Running a pilot survey can help you to check whether
- Your questions are easy to understand and answer.
- You have included all the common options.
- Your survey is too long or too short.
- There are any typos.
It's also good to see how long it takes people to answer your survey so that you can give this information to your future respondents.
Running a pilot survey can help you to identify any problems that your survey might have before you administer it for real.
Check Your Survey Asks What You Want it to Ask
Give your survey one last check over.
Do the questions match up with your original aims and objectives?
Are there any questions that aren't needed?
Will your questions help you to find out what you want to know?
Once you've completed all these steps, your survey is ready to go.
Go out and ask your questions, and have fun analysing your data. Don't forget that report at the end!