A Researcher Who Became a Respondent (Part 3): Super and Super Awful Survey Questions

Editor’s Note: This is the third installment of A Researcher Who Became a Respondent. Read part one and part two.

About six months ago, I saw an ad on Twitter for paid surveys and eagerly joined several online research panels. Since then, I consistently set aside time every day to take surveys in the name of research. Despite the ups and downs, it has been a fascinating journey that has helped me grow as a researcher.

One of the biggest “Aha!” moments for me was when I realized the wisdom of decoupling survey length from survey engagement. As researchers, we seem to be under the assumption that a survey under twenty minutes that is mobile-friendly will keep participants engaged and committed. But the reality is that brevity alone does not automatically yield a high engagement level. In fact, short surveys can still feel painfully long, tedious, and result in a high drop-off rate which makes the study more challenging to field.

What makes a survey question good?

While it is still true that the tenets of good design keep surveys short and mobile-friendly, there are far more factors to take into account when creating good survey questions. Having a  “participant-first mindset” is key to improving the survey experience as a whole. The infographic below illustrates this point with five super survey questions that make the survey experience better, and five super awful questions that spoil it for everyone.

Much like TV shows and Tiktok videos, length is no indication of how engaging the experience will be. Short surveys are great, but they’re no magic formula for improving survey engagement.

As researchers, we need to balance our survey research needs with the participant experience. We know some surveys are more demanding by nature (e.g. conjoint, segmentation, etc) and those will not go away. But, having a “participant-first mindset” is key to achieving more thoughtful designs that will improve response rate and yield a high engagement level.