Social Media versus Online Panel Sampling: Not All Respondents are Created Equal

Rockbridge conducted an online survey using respondents from a leading online panel, as well as those recruited from a popular consumer social media site. A comparison of results reveals striking differences in representativeness and quality.

Social media has experienced significant growth in the past few years and now appeals to a mainstream audience. For many, this means having to decide whether or not to accept their mom's Facebook friend request, but for market researchers, there is an opportunity to achieve a more inclusive sample by expanding the universe beyond that of client lists and online panels.

To evaluate the effectiveness of social media as a sample source, Rockbridge conducted an online survey using respondents from a leading online panel, as well as those recruited from a popular consumer social media site. A comparison of results reveals striking differences in representativeness and quality.

• Panelists are more likely to have a college degree (68%) than respondents from the social media site (40%). Respondents from both sampling frames have higher educational levels than the national average reported by the Census Bureau.
• Panelists have a higher average annual household income ($87,000 compared to $59,000 for the social media sample). Again, both are higher than the national average.
• Panelists are slightly older (mean age 49 years compared to 45 years for social media). Both are older than the national average.

It seems that online panels are a better source for studies that target a more educated and affluent population. If the study requires a target population of less educated, less affluent or younger respondents, the social media source is more representative. The considerable differences between respondents suggest that there is an opportunity to be more inclusive by utilizing both sample sources, which could ultimately help create a more representative sample. Note that in most studies, Rockbridge weights the sample to match the general population characteristics in the Census

Although social media presents an opportunity to sample from a larger and more diverse audience than with online panels alone, it also has a drawback — quality. Certain survey respondents are considered undesirable for research studies, including those who take too many surveys (hyperactive), are not fully paying attention to the questions (inattentive), or provide dishonest answers. We can identify these respondents through quality checks that are part of our SafeSample™ system, a series of diagnostics that detects and flags problem respondents.

In this comparative study, Rockbridge found that the proportion of respondents who failed to pass quality checks was more than twice the rate among the social media sample than the panel sample. It appears that panelists are more committed to thoughtfully responding to questions as part of their agreement to participate in research.

Social media has clearly matured beyond its college-aged roots. And although the diversity and accessibility of users makes social media particularly attractive as a research tool, there are limitations that must be considered. Through comparative online studies and proprietary methodologies, Rockbridge is making it easier to separate insight from distraction.

For more information on Rockbridge Associates you may contact Gina Woodall, SVP at 703-757-5213 ext. 11 or gwoodall@rockresearch.com, or Charles Colby, President, at 703.757.5213 ext. 12 or ccolby@rockresearch.com.

Tweet

Why Should My Company Care About Being “Green”?

by Joe Bates, Vice President, Rockbridge Associates

The race to be “green” is heating up as never before. It is impossible to watch television or read the news headlines without seeing something related to green products and marketing. In their annual National Technology Readiness Survey, Rockbridge conducted a “green” segmentation uncovering segments that are demographically unique and targetable. Read Article »

Using TURF Analysis to Define a Short List of Features

by Gina Woodall, Senior VP, Rockbridge Associates

Design decisions are typically affected by two conflicting constraints: not enough versus too many. There is a need to identify and include all the necessary features to make a product or service valuable to its consumers. However, there is also a need to avoid an excess of features, since this can drive up costs, increase “feature fatigue,” and slow development. Read Article »

Optimizing your Product: How Market Research can Guide you on the Most Profitable Design

by Charles L. Colby, Chief Methodologist, Rockbridge Associates

Designing a new product or service is marked by challenging trade-offs in the early part of development. The benefits of richer functionality and superior quality increases the development and production costs as well as the required price to break even. Market research methodologies can create simulations of purchase situations that present trade-offs to consumers. Rockbridge Associates explains how "choice analysis" can facilitate product design from the beginning. Read Article »

Understanding Consumer Decision-Making with Means-End Research

by Gina Woodall, Senior VP, Rockbridge Associates

Market researchers have access to a technique used to identify the emotional triggers that influence consumers' purchase decisions. The technique is laddering and has been around for decades, but may be one of the best kept secrets in the industry. Learn how this research method can give marketers the information they need to connect with consumers and persuade them to buy their product or service. Read Article »

Pricing Your Product or Service

by Gina Woodall, Senior VP, Rockbridge Associates

It's a challenge faced by every company developing a product or service - finding an appropriate and profitable price. The smartest approach uses the van Westendorp methodology to understand the message you are sending with the price itself. Learn how price can affect quality perceptions and product positioning. Read Article »

Social Media versus Focus Groups for Qualitative Research

by Gina Woodall, Senior VP, Rockbridge Associates

Social media research tools have similar functionality to social media sites. This article explores the differences between social media research tools and traditional focus groups. Read Article »

A Brand Development Model: How to Define and Measure Brand Equity

by Charles Colby, President, Rockbridge Associates

A Brand Development Model describes the stages of development for a brand, identifies how to measure progress, and specifies the marketing priorities required to reach a higher stage of brand developement. Read Article »

The Path to Excellence and the Net Promoter Score

by Charles Colby, President, Rockbridge Associates

The Net Promoter Score has generated considerable interest as a tool to measure customer satisfaction and loyalty. This methodology relies on a single survey measure, while Rockbridge's Path to Excellence goes beyond the single-measure approach to find actionable indices. Read Article »

Is Customer Satisfaction Worth the Effort? How Linkage Analysis Validates the Impact on Your Bottom Line

by Charles Colby, President, Rockbridge Associates

Linkage analysis measures the impact of customer satisfaction on the bottom line. By linking satisfaction survey information with customer behavior, it is possible to validate the benefits of measuring and managing satisfaction. Linkage analysis also guides the allocation of resources. Read Article »

Market Segmentation Studies: 5 Ways to Get the Most Out of Your Efforts

by Gina Woodall, Senior VP, Rockbridge Associates

What can researchers do to help clients gain a more intimate understanding of the different segments of their market? Read Article »