Harnessing AI’s Potential for Market Research

Artificial intelligence is now being applied widely in the practice of market research. With its ability to identify patterns and create meaning from large, unstructured, data sets, AI is being applied in many different ways. Some of the most common applications include:

1. Customer journey analysis: For online and service companies that have detailed data on customer interactions, AI opens the ability to analyze customer journeys, identify cross-sell opportunities, and identify likely defectors.
2. Advanced, real-time analytics: AI’s ability to parse large data sets, identify patterns, and identify changes in trends is empowering sales, brand, and advertising tracking to be more responsive and provide alerts in real-time when noteworthy changes occur.
3. Natural Language Processing: Properly trained AI can extract meaning from open-ended data, from empowering chatbots, sentiment analysis, and analysis of verbatim comments in surveys, making it far easier to identify common patterns and highlight specific findings.

My own exploration of the use of AI for market research began nearly a decade ago. And honestly, it seems far longer. The pace of change has been dramatic, with new capabilities and applications coming online all the time. My first foray into the world of AI investigated its ability to produce automated summaries for pre-testing, this led to exploring its application to ad tracking, and then back to pre-testing. These experiences have convinced me that AI will play a pivotal role in the future of market research.

Skepticism limits the adoption of AI

Many market research practitioners are wary of adopting AI into their day-to-day practice. Whether because of the rate of change (will I end up with an outdated approach?) or because so much is claimed for AI (it can’t all be true, can it?), or because it is such a different approach to traditional methods (it’s really a black box, isn’t it?), it can seem like a big leap of faith to adopt a totally different approach. Yet, I firmly believe there is great potential in AI, and encourage people to harness that potential for their own companies.

AI must be trained on a large dataset

At its heart, AI is about pattern matching, the ability to parse huge datasets to identify underlying patterns and then match new data to those patterns to make a prediction. To make accurate predictions, an AI must be trained on a big enough dataset. When we set out to create Link AI at Kantar, we used all the data on the 230,000 ads that we have tested over the years, which equates to nearly a billion data points. So, step one in your AI journey is to make sure that the dataset being used to train the AI really is big enough to support the use of machine learning.

AI is only as good as the quality of the data on which it is trained. We have all heard the horror stories of AI’s that have ended up biased in some way. The advantage of applying machine learning to a dataset based on surveys is that it leverages all the learning and methodological best practice built into the original survey instrument. So, the use of a proven, validated survey approach as the foundation of a market research solution ensures you can be confident that the patterns the AI is using are representative and meaningful.

Apply AI in the contexts where it adds value

In applying AI to ad pre-testing, it does not necessarily replace existing survey-based methods but offers numerous new benefits above and beyond a survey-only approach. In that context, here are some applications that have proven to add new value.

• Testing content at scale: 33% of marketers claim to pre-test just “a little” or “none” of their ads each year, with cost and speed to market being the most cited reasons for not testing. When creative is the number one driver of effectiveness, particularly in digital media, this represents a huge opportunity cost in media investment spent against potentially ineffective content. An AI-based approach offers a solution to that unmet need by largely removing the barriers of speed and of cost. For instance, Google used Link AI to assess 11,000 ads in one month and identify the impact of 180 creative features on both short and long-term outcomes, a project that would have been impossible using traditional survey methods.

• Screening out ineffective content: In the context of pre-testing, I believe that the use of AI will have the most impact on digital advertising effectiveness because it opens new opportunities to screen out weak content before it gets used. The traditional approach of in-market A/B testing requires large samples for test and control cells if it is to produce reliable results, so pre-testing jump-starts the ability to identify effective campaign pool outs and to improve the overall quality of a brand’s digital advertising.

• Mapping the competitive advertising landscape: Ensuring that your brand’s content is distinctive and stands out from the competition is essential, but due to budget constraints brands very rarely assess what works and what does not in their own category. An AI-based approach allows a brand to test both traditional and digital video advertising to assess how competitive advertising might resonate with consumers. And when a competitor launches a new ad campaign, AI facilitates getting a quick read on how effective that might be.

Survey-based methods have an important role

No matter how fast an AI-based approach might be, there are still specific needs that require a more customized approach, and there are some things that AI cannot take account of yet. If you need more nuanced feedback, then you need in-depth survey-based reactions from the target audience. And when it comes to some topic areas, an AI will not be able to give accurate feedback on its own. For instance, while AI can identify when people are present in an ad, it cannot recognize celebrities and assess their value within a specific execution.

Using AI creates greater flexibility

Rather than replacing survey-based market research, AI-powered solutions offer an additional tool in the toolbox. In the context of advertising development, I believe that effective, early-stage development requires an in-depth assessment of potential campaigns, but an AI-based solution can then provide the last check on likely effectiveness before content is run. And I believe that we’ll only see the power and capabilities of AI expand in the years to come, which promises to gain even more insight from data that is already available but not easily analyzed by traditional means.