Survival of the Fittest: A Glance Into the Future of Market Research

The aim of this post is not to address the survival of market research companies within the industry, but rather the evolution of the industry as a whole, and the survival (or extinction) of current methodologies, and the birth of new ones. Some well-established methodologies will become extinct, while others may survive by evolving and adapting to the new world order.

History has repeatedly proven that extinction comes from a lack of willingness (or ability) to adapt to change – this applies in nature, in business, and in society. At the same time, nothing (not even the dinosaurs or the Titanic) is too big to fail, and this includes Market Research as we know it today.

When it comes to customer insights, there are 3 main sources of data:

1. Asking – in online focus groups or short intercept surveys
2. “Listening” – analysing public data posted by consumers online
3. Tracking behaviour such as retail purchases or website visits or even online searches

Starting with asking questions, which currently accounts for the majority of the US$ 65 Billion that is spent on Market Research globally each year, this can be either quantitative or qualitative.

Several aspects of traditional online surveys are problematic, for example relying on respondent memory or recruiting “professional respondents” from access panels, or the fact that people lie to look good. Short intercept surveys may be the answer to all these issues. The underlying principle is quite simple: if you want to get a consumer’s opinion about something they did or purchased, ask them as soon as possible, and keep it REALLY short!

At the same time, online communities make it possible for a more flexible way of asking questions and engaging with consumers. One could start out with a poll question, follow with a task to upload relevant photos and one or two open-ended questions as part of a bulletin board, then conclude with a chat group for the top contributors. Online focus groups allow for people around the world to essentially “sit at the same table” to discuss and collaborate in an asynchronous manner.

Moving on to social listening. Well for starters, it should have sentiment and semantic accuracy over 80% to be useful. Machine learning (A.I.) is the way to achieve this. Market research purists will ask “how representative is the respondent sample?” – a question that could not be any less applicable to this data collection method. Why? Because this has nothing to do with a sample (we collect all the posts about a brand) and these are not respondents! Some fellow market research practitioners even believe that only negative opinions are posted online, creating a bias for the methodology as a whole. This is actually not true as online sentiment is more often than not found to be predominantly neutral, but even if it were, insights professionals should still have an interest in this data source, as the majority of consumers that according to them do not post online are still impacted by the opinion of those who do.

What about tracking consumer behaviour? Well, certain types of behavioural data such as retail sales or Google search can easily be integrated with social and survey data. Based on DigitalMR’s findings, there is a high positive correlation between retail sales and consumer sentiment as found in posts online. In a project conducted by DigitalMR and Nielsen, consumer sentiment correlated with FMCG product sales at R²=0.81, confirming that brand sales are impacted by sentiment expressed online. Another example is how the brand share of Google searches can enhance a composite index that comprises of multiple social listening metrics and answers the question “how much of my media budget should I spend on Social Media?”

So what should we expect from Market Research in the future? I think we can expect that – despite the common belief that what I am about to say is against “the laws of physics”: it will be better, faster, and cheaper, becoming affordable to organisations of various sizes. Technology-driven methods such as text and image analytics using A.I. and social listening, online communities and short intercept surveys will finally be adopted by the majority of market research practitioners, who will use them to their advantage, combining data sources to synthesize actionable insights for their clients.