Categories
December 23, 2021
Reflecting on how we can improve survey design for respondents.
On this most sacred night of Festivus, I come to you, a humble survey respondent, my aluminum pole in hand, for the traditional Airing of Grievances. I got a lotta problems with you people, and now you’re going to hear about it!
So many ways you’ve disappointed me over the year. Where to start?
Let’s start at the starting. Have you ever tried to take a survey you’ve been invited to?! I give my age, gender, location, and shoe size, and then get a different looking website asking my age, location, and the size of my bank account, only to then get yet another website asking my age, location, and my car’s model year, only to be told I don’t qualify for the survey. Of course, I don’t qualify for the survey. You made me put in a model year – but I don’t own a car. If Google can track me across a hundred websites, and Apple can track me across a dozen apps, can’t you at least remember my age, location, and gender, all of which you made me tell you when I signed up?
I’m told this endless parade of repetitious questions is called a “screener”. While I realize commercialism is against the true spirit of Festivus, you should pay me for these screener questions, whether I have screens in my apartment or not. These screeners take a lot of time, and they are as boring as this tinsel-free aluminum pole.
So I break through the screener and get to the good part. The actual questions of your survey. Who writes these?!
First, I’m told that I have to verify that I’m not a robot. Then you ask me to assign a number to everything like I am a robot. On a scale of 1 to 5, how satisfied am I? 1, I guess? You then give me 100 points to assign how important each thing is to me, while leaving out the thing from the list that is actually important to me, then force me to add everything up to 100 like the world’s most boring sudoku. No, I can’t put a precise dollar value on having 500GB instead of 200GB. On a scale of 0 to 10, how likely am I to recommend this particular medicine? You think I go around recommending hemorrhoid medications?
Speaking of robots, can you stop being robots and start to realize we’re not all the same as you? I don’t have a job in an office. I don’t sit at a desk. I don’t own a car. Stop assuming that I do. I take the subway to the job site, and I carry a toolbox, not a laptop.
Where were we? Oh, questions! I don’t know which is worse, the essay questions or the lists of choices. Either way I get flashbacks to high school tests. The lists of choices always leave out the answer I would want to select. At least include an “Other” box for me. Instead, you require an answer for me to proceed, and I have to pick one that doesn’t quite fit at random. And the essay questions – look, it’s bad enough I confessed to your survey that I bought hemorrhoid medication, do I really have to write a review of it? I don’t want to write 100 words about it, so don’t get mad at me for copying and pasting the question text over and over and over. What a pain in the ass.
Over and over and over you ask me different versions of the same question. And stop it already with the grid with 10 items I have to rate. I was once asked to complete a grid with dropdown boxes in each column instead of a button. It was like 100 questions on a single page. If I wanted to do multiplication tables, I’d help the kids with their homework. Anyway, I couldn’t rate 10 different brands on 10 social policies they each held, so I quit that survey.
Look, I guess your customer knows why there are 20 different versions of their product on the store shelves, and knows everything about how they differ, but I can’t remember which of these 20 I bought. I don’t even understand what some of these words mean. Should I be looking for bits with “optimized torsion zones”? I’ve probably been using unoptimized torsion zones my whole life and never noticed. No, I can’t tell you the exact number of tools I have in my toolbox, nor the brands of each tool in my toolbox. Heck, I don’t even remember the brand name on the toolbox, and I look at that every day.
Now about those incentives. They’re way too low, and they take forever to be paid. I don’t know why I even take these surveys sometimes, given the meager amount they pay. But I’m sitting at the job site having lunch and I’m tired of my sudoku app and I don’t want to read what my uncle just posted on Facebook. So I let you annoy me to death instead.
You know what would be a Festivus Miracle? If you showed me a little empathy. Stop asking me questions that are worded the way your customer wants, and ask me questions that I will understand, questions that I can answer fully and completely, and in 10 minutes or less. Don’t assume your questionnaire will get you all your answers. From this side of the smartphone, it looks like you need to do some research yourself beforehand to figure out what questions to ask me in the first place!
Anyway, if you want to annoy me with one of your surveys again, you’re going to have to wrestle me for it. It’s time for Feats of Strength.
Comments
Comments are moderated to ensure respect towards the author and to prevent spam or self-promotion. Your comment may be edited, rejected, or approved based on these criteria. By commenting, you accept these terms and take responsibility for your contributions.
Disclaimer
The views, opinions, data, and methodologies expressed above are those of the contributor(s) and do not necessarily reflect or represent the official policies, positions, or beliefs of Greenbook.
More from Jeffrey Henning
Some “tips” so that you can continue to enjoy writing, while pretending to simply be an editor of AI-generated content.
Given the diverse backgrounds of market researchers, there is a real need to continuously train.
MRII’s survey on how the market research industry is doing in career satisfaction, growth opportunities, and learning preferences.
Jeffrey Henning details the 10 most retweeted links shared using #mrx over the last two weeks.
Sign Up for
Updates
Get content that matters, written by top insights industry experts, delivered right to your inbox.