If the Questions Don’t Matter to Your Customers, the Insights Won’t Matter to Your Business
If the short survey is still asking consumers questions that don’t matter to them, then the answers will still be distorted by our respondents’ boredom. What can we do as researchers to improve the situation?
Many of us in market research, especially those in methodologist and data science roles, come from an academic background. We’re trained to think deeply about the implications of our assumptions for our statistical models. We test and design instruments, develop best practices, and deploy surveys, using the tools of social science to mine data and insights for our corporate partners. But in all of this high-minded science, we’re missing some simple lessons about survey design that may lead us, and our clients, astray.
To put it simply, too many of our questions don’t matter to our respondents. We’ve worked for years as an industry to make surveys shorter and our respondent experiences better, to develop better models and statistical techniques. But if our respondents simply don’t care about the answer to our questions, then we can never be sure about the quality of our data. Rather than thinking about respondent engagement as a user experience design challenge to be solved by programmers, survey writers and other researchers need to take responsibility for asking our respondents questions that matter to them.
When researchers pile on objectives, modules, and attribute batteries, we’re taking up valuable survey real estate, but we’re also taxing the energy our respondents have to care about their answers. I may have strong opinions about the first 2 minutes of questions about a grocery category, but at minute 5 there’s a good chance I’ve stopped caring about which frozen pizza brand is the most “inquisitive.” Most likely, the responses in the aggregate will likely follow the same general trend as earlier, more “high-level” questions, rewarding the most popular brands in proportion to their competitive standing.
One often-used solution to this problem is to make our surveys shorter. While this can work to a point, long surveys are only a small part of the issue. If the short survey is still asking consumers questions that don’t matter to them, then the answers will still be distorted by our respondents’ boredom. If bored, disengaged respondents are giving us bad answers, what can we do as researchers to improve the situation?
• Ask people about themselves – a common piece of advice for someone on a first date is to ask your companion a lot of questions about themselves. Everyone loves talking about themselves – it’s a topic of interest, and we’re all experts on it. From a marketing standpoint, these answers allow brand wranglers and marketers to understand their brand and category from the consumer’s perspective. Instead of thinking about which messages will resonate as messages, we turn our perspective to one of “how can we craft a message that resonates with our potential customers as they already live?” Focusing on the consumer provides a better experience, keeps the user engaged, and ensures that the answers are grounded in the life experiences of the customers we’re trying to cultivate.
• Avoid brand narcissism – if we’re asking a long series of questions about our brand, we’re likely boring our respondent to death. Even if our respondent purchases in our category, they don’t care nearly as much as we do about contrasting competing brands. By focusing solely on our brand and competitors, we’re engaging in “brand narcissism,” forcing respondents to act as if they care just as much as the folks whose careers and livelihoods depends on the brand’s success. Instead of having the conversation solely on the brand’s terms, allowing customers to frame it within their own experiences leads to actionable insights that reveal the best touch points to activate for different consumer groups.
• Focus on what matters to respondents –It’s time to give up on the idea that our surveys must have a direct question for each potential objective or point of interest. Rather than asking everyone about everything, choose to focus your survey questions on the attributes that your respondents identify as important. In other words, figure out what aspects of your product/brand/category are the most salient to the respondent, and then follow-up on those specific issues. This allows us to free up survey real estate by moving more inferential work to the back-end, and ensures that we’re not over-interpreting issues that may have little meaning for actual customers.
Consumer survey research is both art and science. While we’ve done a good job of building up the science-side of our work, ensuring validity, reliability, and representativeness, we’ve often given short shrift to the ways in which our surveys are received by respondents. By artfully crafting questions and survey paths that matter to the end-user, researchers can obtain a more engaged respondent experience and more actionable insights for their clients.