Is Market Research Ready for Big Data?

By Ray Poynter

Big Data is set to be the biggest disruptive change that market research has seen since the arrival of the telephone, maybe since the arrival of the computer.

The ESOMAR Global Market Research report 2010 shows that about 50% of the revenue that is currently nominally called market research comes from activities that do not relate to asking respondents questions (i.e. they do not relate to surveys, focus groups and the like). This 50% relates to store audits, people meters, processing loyalty card data, web analytics and other data related services. This 50% is the thin edge of the wedge for the Big Data transformation that is about to wash over marketing, market research, Business Information, CRM, Governments, and citizens.

The term Big Data refers to the integration and utilization of the hosepipe of data being generated by companies, governments, social media and people themselves. Two views of big data are beginning to emerge from market research perspective, although there is a considerable amount of overlap between these two views. The first is a brand-centric view, where all of the data held by a brand, e.g. the CRM data, loyalty card data, market research data, and data sourced from social media are integrated and utilized. The second view is a third-party data warehouse model. In this second model, companies, for example a panel company, might access a wide range of information about people and utilize that for multiple brands and clients.

In both the brand-centric and warehouse model of Big Data, explicit questions (for example surveys and focus groups) will play a smaller and smaller role, as the databases are able to provide more information about what people do, based on observation as opposed to recall. The threat to market research is that people will feel that having enough data will remove the need for market research altogether. The opportunity is that market researchers can focus on putting the “Why?” into the picture, and thereby make the “What?” that Big Data can offer more valuable.

The move from traditional market research to Big Data is likely to be a bumpy and not altogether pleasant one for many market researchers. It is likely to require a fundamental change in codes of conduct, moving away from market researcher’s creating codes to models based on informed consent. It will also require market researchers to understand the power of Big Data and to become comfortable with the tools and techniques of Big Data.

On 20th March Katie Kaylor from Phones4u and I will be presenting a paper to the UK MRS Conference looking at how community panels will change over the next five years as they converge with Big Data. We will also be broadcasting this as a webinar on 12th April.