Surveys. I have a lot of feelings about surveys and my instinct is that the all encompassing “we” (libraries, higher education, businesses, politicians…) have a habit of throwing surveys at our questions. How often have you heard the phrase “Oh, we’ll just do a survey” in your professional life? When was the last time you took a survey (or saw and ignored one) that was sent to your email or popped up on a website or even came in the mail? They are everywhere.
A survey is often the default or "go-to" research methodology when one needs to answer questions about what people, like, expect, or want, among other things. While surveys are likely to be considered the easiest option -- you can “throw one together” in Google Forms, Survey Monkey, Qualtrics, etc. -- you can’t conflate “easy to create” with “easy to create well.” Crafting a good survey is HARD.
Even if a survey is the appropriate methodology for the question you’re looking to answer, the questions you ask, the way you ask them, and the options you give people for responding all require a thoughtful approach, rooted in how the real life people involved in survey creation, participation, and analysis contribute to its utility and effectiveness.
Whither the survey
First things first. Let’s talk about whether you should even do that survey. If you catch yourself thinking “I’ll just do a survey,” I heartily encourage you to pause, reflect, and think critically about whether a survey is the best mechanism for what you want to learn. Here are a few prompts to think through:
-
Can you get a meaningful group of people to take the thing? In at least some loose terms, think through whether you can capture a representative sample. If not, it will be hard to argue the validity of your findings. Thinking about how targeted your audience is can also be helpful.
-
Is what people say enough or do you need to know for certain what they do? The only way to truly know what people do is to watch them. A survey is never going to get you that.
-
Do you need to find out the who, what, and/or when? Or do you need the how and why? Surveys are much better at capturing things like who, what and when, then they are at how and why. How and why are behavioral, so this gets back to whether or not what people say is good enough.
-
Do you want to learn about past experiences or expectations? For the same reasons mentioned in the points above, you’re much more likely to get a decent picture of things that already happened than those that might.
What questions to ask
OK. So you’re going to do it. You’re going to create the survey. Every question you ask on a survey should serve a clear purpose.
I always come back to this great prompt from Erika Hall in her post On Surveys: “Will the people I’m surveying be willing and able to provide a truthful answer to my question?” That “truthful” is so key. Are you asking something that would be super difficult for someone to remember? Is it something where they may feel influenced to provide a particular answer? Are you asking them to predict the future? If you have to stretch at all to justify whether someone can answer a question truthfully, you probably shouldn’t be asking it.
Some other things to consider:
-
Are you prepared to respond or do something after hearing the answers to the question? This is especially so if you’re asking for feedback.
-
Why are you asking the question? We often collect data just to collect it. Everything we ask should be for a reason.
-
Who within the organization will use the answers?
-
What will those people use the answers for?
-
Will questions be required or optional? And keep in mind that you don’t want people to have to lie to get through your survey. Questions should only be required if they really are required.
-
Are you asking something where the participant might wonder why you need to know it? It’s ok to take a sentence or two to explain. Be honest with your participants. Be human!
Come back for the rest!
Part Two of the post will discuss how to ask your questions and what types of questions to use, as well as provide a prompt for practicing your newfound survey creation powers.