Take Control of Your Global Online Sample

Common concerns among global tracking studies include unexpected changes in your tracking data resulting in inconsistent samples. Recent work being done by researchers provides a new way of conceptualizing online sample consistency globally: analyzing purchase intent. Learn more in this article from The Operandi Group.

Congratulations, your client finally approved that huge multi-country tracking study. It’s a big, hard-fought win for your company. Pop the champagne!

Okay, so you haven’t done much work in Eastern Europe or Asia - at least not online work - but your team received a few quotes from panel companies, and they all said it could be done.

Will you sleep well tonight? How about when fieldwork starts?

Interesting news - Panel Company X changed ownership. Wait, aren’t they the ones you’re using for your new global tracker?

More news – Panel Company Y announced a new recruitment sourcing partnership. Aren’t they backstopping Panel Company X on that huge new project?

You get the picture.

These are common concerns among those conducting multi-country online research. Online sample can be a black box, even more so internationally.

The big fear is finding an inexplicable change in your tracking data and not being able to rule out the effects of inconsistent sample. That’s when you’ve got some serious explaining to do.

If you share these concerns, I urge you to take a look at the work being done by Steve Gittelman and Elaine Trimarchi at Mktg

Steve and Elaine have been collecting data across a wide range of panels since the fourth quarter of 2007. Their data includes samples from over 180 panels across 35 countries. They call their effort the Grand Mean Project.

The basic idea is this: there is no gold standard for the composition of online sample. The days of comparing phone data to Census data are over. So, given the rapidly -changing nature of online sample sources, there is a need for some standard to ensure consistency both within and across panels.

Steve and Elaine reason that in the absence of a gold standard for sample composition, the most important thing to measure is consistency of results. In other words, the soup may be made slightly differently each time, but does it taste the same?

Their questionnaire takes measures of sample composition such as demographics and professional respondent metrics. But importantly, it also measures purchase intent across multiple domains. The data for each country is analyzed separately.

Mktg’s data is a much-needed canary in the coalmine for changes in the metrics ultimately most important to most clients – consistency of purchase intent. The data allows researchers to make informed decisions about sample composition and possible sample blending solutions.

Participation by online sample sources in the Grand Mean Project is voluntary. However, Mktg does publish information on the participation of, and the consistency results of, sample sources on the Grand Mean Project section of their website.

Whether you have an international study in field or are planning for one in the future, the Grand Mean Project provides a tool for managing online projects which previously was not available.

This article was originally published in MRA's blog The Researcher's Perspective in January 2010.

This content was provided by The Operandi Group. Visit their website at www.operandigroup.com.