In 1883, the eminent English biologist Thomas Huxley, stated to the world that overfishing of sea fisheries was “scientifically impossible” and dismissed calls to address the fishing techniques being utilized. At the time, Huxley was considered an expert of his time on many things, most notably as one of the key supporters of Charles Darwin’s theory of Evolution. He couldn’t have been more wrong about the fish stocks but this didn’t stop this school of thought continuing well into the 20th century.You may ask why I am writing about fish. The reason, is that this is the best possible comparison I have found for how we currently consider our research respondents. It is important to remember that a large proportion of our industry is based upon the willingness of the public to provide us with their opinions and habits, yet these people are consistently forgotten when we consider the research we are planning to conduct. We simply consider that there will always be “more fish in the sea” for our research even if it happens to be too long, too repetitive or hard to navigate. In hindsight, when Huxley made his speech to the International Fisheries Exhibition, it is now widely accepted the fish stocks were already collapsing and some have never recovered since.
As an industry, we have to remember that sustainable fishing for us, takes the form of a continuous resource of willing respondents who are prepared to contribute to our investigations.
In the 90’s and 2000’s we witnessed a significant decline in willingness of respondents to participate in telephone research – a decline from 36% in 1997 to 9% in 2012 according to the Pew Research Centre – and our reaction as an industry was not to focus on the drivers of the decline but instead to switch data collection to online methodologies.
This meant that we never really considered how to adapt to, and optimize, this medium and we continued to recreate the errors of the past, forgetting the respondent who was at the other end of this data collection technique. What made this even worse in online was that we no longer had an interviewer to encourage our respondents through the toughest parts of the surveys. We lost our personal link to the respondent and this has led to them feeling an even greater disconnect with our purpose, and their role in market research. Unsurprisingly, we have now seen the online response rates decline just as sharply as telephone did - with a drop, in the US, from more than 40% in the early 2000’s to under 10% today in most panel companies I consulted in recent weeks.
Unfortunately, the pace of change and pricing pressures in the industry mean that we have focused on becoming experts in how to catch/recruit our respondents but we still know very little about them.
For an industry which is exceptionally good at helping our clients understand their consumers and drive loyalty, we still know very little about what drives our respondents to come to us, and hopefully stay with us. Some good work has been done on this over the past few years by people such as Pete Cape (Join the dots), Ray Poynter (Future Place) and Betty Adamou (RTG). Unfortunately, very little of this is regularly used in the industry to improve the research respondent experience on everyday surveys and so its true impact has remained marginal.
Why is this so important? Well we need to go back to the fishing analogy and ask ourselves the fundamental question – “In the future, will we have enough willing respondents to answer our research?”.
About 3 years ago, I presented a paper at the MRS Online Methods conference that demonstrated the clear link between negative survey experiences and the decreased likeliness of a respondent to answer the next invite they received. This showed that panels are continuously eroded by respondents facing unsatisfactory survey experiences, and much of this responsibility rests with the questionnaire they are being asked to complete.
Whilst online panels have evolved and mostly kept pace with technology, the survey itself is the one part of our process that we have not significantly changed – and the part that respondents are, in my view, genuinely becoming tired of.
• In a world where multitasking is prevalent, and attention spans short, we persist with 25-minute surveys.
• In a world where micro blogging is king, we persist with question texts that run into multiple paragraphs.
• In a world where the most successful web properties are simple to navigate and get you to the objective in as few clicks as possible, we continue to create mundane, complex, repetitive online forms.
We simply must adapt the way we ask the things we need to ask and think carefully about the interactions between the respondent and the survey.
Now this leads to an inevitable leap to gamification, which remains a hot topic. However, it is often neither appropriate nor possible for many research designs and can present problems with broad audiences. This does not mean that we should ignore its key principles, or those of basic web usability, as they remain essential to driving respondent engagement and are crying out for broader adoption.
At Schlesinger Interactive, we have shown that these principles can be applied to almost any online project to make it a measurably better experience for respondents. We have also shown that this not only drives better panel health, but also better data quality.
I am regularly struck by the significant difference in the way a respondent is treated when we do actually meet them. We host multiple focus groups/interviews daily, and when I am in our facilities I note that they are welcomed with coffee, treated like we need them, talked to like we value them, made to feel they have been a key part of a research project and they often leave happy with their contribution, and reward. In online quant, how often can we truthfully say that our respondents have left a survey satisfied and content with their contribution?
These are all features that are sorely missed in the lonely world of self-administered online surveys, but that we can clearly learn from and apply, bringing the principle of a 2-way respondent relationship back into data collection.
This means that, regardless of the sample source, we simply have to ensure we keep those we have, re-engage those we have lost and engage better with those we have yet to meet. This can only happen through a fundamental re-think of how we question them. Think about how we, as an industry, advise brands to regain lost customers; you have to convince them something has changed and follow through on that promise.
This is what sustainable fishing looks like in online research, caring about the people who make our product – not just selling the product itself.
So back to the key question, are there really more fish in the sea? Well only if we start to take better care of them, rethink the research we put in front of them and ultimately, treat the respondents as if they were the prize catch.
Schlesinger Group, New Jersey
- Iselin, New Jersey
- Outstanding recruitment for any methodology including online surveys. Worldwide project management with 26 facilities across the US and Europe.