Take Control of Your Global Online Sample

Common concerns among global tracking studies include unexpected changes in your tracking data resulting in inconsistent samples. Recent work being done by researchers provides a new way of conceptualizing online sample consistency globally: analyzing purchase intent. Learn more in this article from The Operandi Group.

Congratulations, your client finally approved that huge multi-country tracking study. It’s a big, hard-fought win for your company. Pop the champagne!

Okay, so you haven’t done much work in Eastern Europe or Asia - at least not online work - but your team received a few quotes from panel companies, and they all said it could be done.

Will you sleep well tonight? How about when fieldwork starts?

Interesting news - Panel Company X changed ownership. Wait, aren’t they the ones you’re using for your new global tracker?

More news – Panel Company Y announced a new recruitment sourcing partnership. Aren’t they backstopping Panel Company X on that huge new project?

You get the picture.

These are common concerns among those conducting multi-country online research. Online sample can be a black box, even more so internationally.

The big fear is finding an inexplicable change in your tracking data and not being able to rule out the effects of inconsistent sample. That’s when you’ve got some serious explaining to do.

If you share these concerns, I urge you to take a look at the work being done by Steve Gittelman and Elaine Trimarchi at Mktg

Steve and Elaine have been collecting data across a wide range of panels since the fourth quarter of 2007. Their data includes samples from over 180 panels across 35 countries. They call their effort the Grand Mean Project.

The basic idea is this: there is no gold standard for the composition of online sample. The days of comparing phone data to Census data are over. So, given the rapidly -changing nature of online sample sources, there is a need for some standard to ensure consistency both within and across panels.

Steve and Elaine reason that in the absence of a gold standard for sample composition, the most important thing to measure is consistency of results. In other words, the soup may be made slightly differently each time, but does it taste the same?

Their questionnaire takes measures of sample composition such as demographics and professional respondent metrics. But importantly, it also measures purchase intent across multiple domains. The data for each country is analyzed separately.

Mktg’s data is a much-needed canary in the coalmine for changes in the metrics ultimately most important to most clients – consistency of purchase intent. The data allows researchers to make informed decisions about sample composition and possible sample blending solutions.

Participation by online sample sources in the Grand Mean Project is voluntary. However, Mktg does publish information on the participation of, and the consistency results of, sample sources on the Grand Mean Project section of their website.

Whether you have an international study in field or are planning for one in the future, the Grand Mean Project provides a tool for managing online projects which previously was not available.

This article was originally published in MRA's blog The Researcher's Perspective in January 2010.

This content was provided by The Operandi Group. Visit their website at www.operandigroup.com.

Custom Online Panels Explained

by Dana Stanley, President, The Operandi Group

Custom online panels provide an efficient way of gathering survey feedback from a regular group of participants. This article looks at the what, why and how of creating custom online panels. Read Article »

Jumping to Conclusions about Twitter

by Dana Stanley, President, The Operandi Group

A research report on Twitter found that the platform would fail as a marketing medium, according to a survey of social media users. This article critiques the questions presented to survey takers and the validity of the report's resolution. Read Article »

Interview: Recruiting via Social Media with Simon Chadwick

by Dana Stanley, President, The Operandi Group

This is a video interview with Simon Chadwick, CEO of Peanut Labs. At a 2009 conference he presented on recruiting for online surveys through social media networks. Learn how one firm recruits and maintains their online communities. Read Article »

Interview: Mobile Surveys with Heidi Dickert of Mobile Compass

by Dana Stanley, President, The Operandi Group

This is a video interview with Mobile Compass co-founder Heidi Dickert regarding surveys on mobile devices: their advantages and possible problems as well as design, technical and technological implications. Read Article »

Interview: Technology Adoption Among Market Researchers Globally

by Dana Stanley, President, The Operandi Group

This is a video interview from the CASRO Technology Conference based on a study of 200 research companies worldwide. The study covered technology trends and providers and the explosion of online qualitative research. Dana Stanley of The Operandi Group interviews Tim Macer from meaning ltd. and Patrick Molloy from Confirmit. Read Article »

Interview: Technology Investment in a Recession

by Dana Stanley, President, The Operandi Group

This video interview is based on a presentation at the CASRO Technology Conference in 2009. Dana Stanley of The Operandi Group interviews Hugh Davis of Greenfield Online who provides examples of ways technology services help cut costs and change business models. Read Article »

Interview: Automating Analytical Protocols in Market Research

by Dana Stanley, President, The Operandi Group

This video interview with David Cosaro covers ways of using technology to convert raw survey data into visual presentations quickly and accurately. It is based on his presentation at CASRO Technology Conference in 2009. Read Article »

Interview: Online Qualitative With Jim Longo of iTracks

by Dana Stanley, President, The Operandi Group

This video interview covers the transition to online qualitative research including initial concerns and discovered advantages. Jim Longo of iTracks details specific ways research companies approach the online space. Read Article »

Interview: Rich Media Surveys

by Dana Stanley, President, The Operandi Group

This video interview discusses preferences of survey takers regarding plain text versus rich media flash-based surveys. The study looked at specific demographics and found interesting results. Read Article »

Animate Your Data

by Dana Stanley, President, The Operandi Group

Dana Stanley of The Operandi Group shares his insights on making your data stand out. PowerPoint presentations can be tedious and boring. This article recommends other types of technology you can use for data reporting. Read Article »

Research Industry Taxonomy

by Dana Stanley, President, The Operandi Group

This is a guide to the basic types of research companies and how they work together. Read Article »

Interview: Kathryn Korostoff, founder of Research Rockstar

by Dana Stanley, President, The Operandi Group

Research Rockstar is an online training company that shares education on marketing research topics. This interview with the founder conducted by Dana Stanley shares insights on the most popular classes, including how to hire the right agency or which research method to use. Read Article »

Interview: Communispace CEO Diane Hessan

by Dana Stanley, President, The Operandi Group

Dana Stanley interviews the CEO of Communispace - a company that uses online communities to find consumer insights. This interview describes the differences between online communities and custom panels, addresses positive bias, engagement, community size and more. Read Article »

Interview: Tom Anderson on Data Mining and Social Media

by Dana Stanley, President, The Operandi Group

This interview with Tom Anderson conducted by Dana Stanley covers the basics of data mining, approaches to making connections on social media sites, and where the research industry is headed. Read Article »

Market Research in the Social Media Age

by Dana Stanley, President, The Operandi Group

This video is of Dana Stanley presenting to the Maine Marketing Association. Targeted at a marketing audience, this presentation covers the evolution of social media and its relation to market research, including revolutionary technology tools and forward thinking. Read Article »

Possibilities for Market Research: GPS Apps

by Dana Stanley, The Operandi Group

Dana Stanley reviews Foursquare, a mobile phone app that uses GPS to note your location and allows you to leave comments and connect with friends. This technology creates new ways for market researchers to connect with consumers in the moment of their purchase. Read Article »

Interview: Online Qualitative Research

by Dana Stanley, President, The Operandi Group

An interview with Jen Mason Drolet, Vice President of Client and Moderating Services at iModerate Research Technologies, about online qualitative research. Read Article »

Creating Surveys on iPhone Apps

by Dana Stanley, President, The Operandi Group

Dana Stanley reviews an iPhone app for creating surveys in this article. Follow the process of creating and posting a survey and learn insights into possibilities for its use in market research. Read Article »