Editor’s Note: I’ve been thinking quite a bit lately about the role of automation, AI, and human capital in the near future of the research industry and specifically what shape the disruptions of technology will take on the human element of research. It appears I am not alone, and as usual Ray has beaten me to the punch in laying out what I think is a very probable (in fact, almost certain) scenario for how these factors will play out over the next few years. His summary below should be taken as prescient, and I know I for one, as well as many of those that I advise and/or work with on various business initiatives, have been making plans that assume this future is in fact how things play out broadly. Take that for what it’s worth as you evaluate his thoughts in light of your own personal or business planning.

This post was written in response to a session at the MRS Conference in London where, earlier today, a panel was discussing AI (artificial intelligence) and seemed to have a mind-boggling degree of complacency about the impact of AI on market research and insight. To summarize their views, machines are good at repetitive tasks but can’t be creative, the jobs that will be absorbed by AI will tend to be those that are already automated, or those done by more junior staff, and those that are repetitive.

However, I think the role of AI will be much more disruptive than the panel seems to think.

Assumptions

A couple of starting assumptions:

• Computers do not just do what they are programmed to do. Computers are increasingly programmed to learn, not to do – that is why we call it machine learning. Computers are already analyzing data and writing reports and have been doing so for a while.
• A large share of current market research is poor; it too often utilizes a poor design, it uses a poor questionnaire or a poor discussion guide, the analysis is not deep enough and does not access all of the available information, and results in a less than impressive report or presentation.

AI will not just make the repetitive tasks faster and cheaper; it will also over a period of time, replace the worst (approximately) 75% of market research, leaving just the best as having a significant human component.

The future?

I predict the following will happen over the next few years (perhaps 5 to 10 years):

• Project Design. Machines can learn the best range of designs for all the common research problems, e.g. tracking, concept tests, U&A, brand equity etc. The programs will ask the users of the research some questions and offer some choices (such as quant or qual – spelling out the implications of the choices in terms of time, cost, meaning etc). For most research projects the questions (e.g. the questionnaires or discussion guides) should NOT be written from scratch; there is lots of research on which questions work best, a machine will tend to use better designs, better questions, and be less likely to miss key elements.
• Project management. The best way to handle problems is to spot them early, that is a great role for AI. Some of the problems found by the AI project manager will require a human decision, but that decision will be asked by the system and the options and their implications will be spelled out. However, in many cases decisions to increase the recruitment, to add new regions, to modify a question or probe will be made by the AI project manager without having to wait for human intervention.
• Analysis. There are already systems that can interrogate data and write reports (check out Intellection for an example). With AI project design and AI project management the AI analysis will be even more powerful; for example drawing on wider information, taking things like cultural values into account, and looking deeper into the data than is normal. It is likely that the recommendations and interpretations generated by AI will be followed by a discussion between humans, but I suspect that in many cases more than 90% of the recommendations and interpretations will be used without being edited.
• Communicating insight and recommendations. An AI approach to communication would factor in (in a formal and analytical way) what forms of communication work best with the message and the target audience. This would include small things like colors, spelling, reading age, but also writing style, idioms, analogies and story structures.

Winners and Losers

So, given my view of the future of AI, who will be the winners and losers?

The Losers

Here are some of the jobs/tasks that will change or disappear.

• Project design. In the future most projects will not require a researcher to determine things like number of interviews, questions, locations etc. AI will generate a design and the end user and an advisor will sense check whether it is right. So anybody designing the detail of a project will see the task diminishing, the placing of fieldwork and/or recruitment contracts will be automated, as will most of the testing of the study.
• Quant data collection. The placement of data collection contracts, the creation of the data collection instrument, and the monitoring of the data collection will be 99% AI.
• Qual data collection. I think focus groups and IDI’s will be largely moderated by AI, but not for a few more years. However, semiotics, text analytics, social media will all become increasingly AI quite quickly.
• Analysis. I see most analysis to be AI-led soon, with the crafting of the recommendations and interpretation falling a little but behind.

Winners

Here are the roles that will, in my view, prosper under the rise of AI.

• Client success managers. These are people who get to know the client’s real needs, help shape the use of research, and help socialize the findings of research into the wider organisation. The people who will flourish are those with people, business, and AI utilization skills.
• Bespoke researchers. Some people want bespoke suits, some need them (if their shape is non-standard for example), similarly there will be clients who want and/or need bespoke research. The people who will flourish here are those whose knowledge, presentation skills, and reputation mean that they can be more effective than AI research.
• Entrepreneurs/Intrapreneurs. These are people who will identify new businesses and opportunities. These people will be ‘outside-the-box’ thinkers who can see new ways of doing things.
• People creating the AI systems. This group includes coders, system analysts etc, but it also includes the people creating the best practices, the people helping the machines learn, and people creating the bridges from people to machine and machine to people.
• The people driving AI systems. In order for AI to design a research project you need to know what it is you really want to achieve. For the foreseeable future the process of answering the questions from the AI systems will require humans, humans who understand what the questions are and who have a sense of when the answers the research user is offering are only partly correct. These people will also know when to override the AI, when to instigate changes in the AI solution, and how to best use the outputs.
• Performers. This is a catchall term for writers, presenters, comedians, storytellers, artists, cartoonists etc who can bring results to life in ways that AI will not offer in the foreseeable future.

My guess is that over the next 10 years 60% of current jobs in MR will go, and perhaps 20% will expand/emerge.

Note, these changes will also impact HR, sales, marketing, IT, finance and indeed every aspect of the organization will also change, with the same sorts of ratios.

How does this picture compare with your thoughts about AI?