Gamification in Market Research – Is Respondent Engagement More Important than Bias?

I’ve just finished reading an article on Gamification in the May 2012 issue of Admap by Deborah Sleep of Engage Research (http://bit.ly/KVuJ97 ). It describes the findings of research carried out jointly by GMI and Engage on the impact on respondent engagement during online surveys.

The main gist of the article is simple: that richer responses can be generated by the integration of gaming mechanics – including feedback loops, tapping into competitive spirit, framing the questions as tasks to name a few. Respondents spend more time in giving answers, enjoy the experience more, give richer responses.

Great stuff – especially for online panel providers, and potentially for all involved including Clientside researchers such as myself.

Yet it left me wondering : does Gamification actually move the perceptual goalposts with its brain-engaging techniques?

Or more critically: does Gamification introduce a new element of bias? Here’s my take:

1. Gamifying is a powerful tool, one that needs treating with care. A gamified question can easily elicit a different response from a straightforward question.

Take an example from the article: a question that originally asked respondents to “name the clothing they liked to wear” was gamified to: “what clothing would you wear for a first date?”

That to me is a different question – I’d expect answers to both questions to be different.

2. Utilizing gamification should always be informed by the actual knowledge needs you have.

If – as this research shows – you get richer response say on brand mentions, then the design needs to consider how important the accuracy of measurement is versus richness of mentions. Richness may not be a relevant criteria, depending on the goal of the survey in hand.

3. Care needs to be taken with rating scales that are part of a gamified survey.

More engaged respondents are likely to give more positive evaluation on rating scales. Where normative data is a key aspect, this is important to be able to understand better – especially if we’re needing to look at purchase propensity and top-box scores.

4. Authenticity of response should be equally important for any survey as engagement. Utilizing gamification needs to take that into account.

It’s potentially misleading to introduce a heightened awareness into a survey context if the real life experience isn’t likely to match that. Not all things in life are a game, or fun – if reality is monochrome, then research needs to replicate that.

My overall take on Gamification is that it looks like a relatively important methodology innovation that needs proper  validation – so the R&D work needs to be done before broad adoption can be advocated.

More particularly, I think we need to understand  what biases we’re introducing, so we can understand what impact what types of gamification mechanics have on response patterns, and build that into analysis.

The upsides in engagement may well be there for MR – but gaining a relatively granular understanding of the impact of change is important.

Curious, as ever, as to others’ views.