Article:
The Real Mystery of Political Polls
by Copernicus Marketing Consulting and Research
Ahead of the 2008 U.S. presidential election, Copernicus Marketing Consulting and Research looked at the phenomenon of "polling failure" and response bias based on social desirability.
We’re getting down to the wire in the U.S. presidential election and all eyes are on the polls. Yet for all the talk about who's ahead and by how much, there’s even more talk about why the results of the polls are not all that valid. After so many embarrassing invalidations of polling work, everyone—from the candidates, to the media, to the pollsters themselves—routinely hedges the numbers with one explanation or another about why we shouldn’t be too surprised if the outcome on election day looks nothing like what the polls prognosticated.
Throughout history, there have been many explanations for “polling failure.” In 1948, for instance, the Readers Digest poll—at the time the most prestigious in the nation—predicted that Thomas Dewey, the Republican, would win big in the presidential race against Harry Truman, the Democrat. Truman, of course, ended up winning and the perception of polling as a reliable resource hasn’t been the same since. In this case, the reason the polls were so off was the unrepresentative nature of the Readers Digest sample. As it turned out, the magazine’s readers and poll sample were more Republican than the rest of the country.
These days most polling firms—including Gallup, Pew, USA Today, Zogby, and Rasmussen—use a relatively straightforward polling technology: they ask people a single question about who they plan to vote for in the next presidential election. They present the options and let respondents decide. They do this on the phone, sometimes in person and increasingly over the internet. Each respondent is then weighted by his/her self-reported probability of voting or estimated probability of voting based on past returns for the region/state/city/zip code and/or demographic profile they exhibit. The more sophisticated the weighting, the more accurate the poll.
While we haven’t come across too many people taking issue with the sampling and weighting of 2008 presidential polls per se, many have wondered aloud about how honest potential voters are with pollsters when it comes to their intention to vote for Barack Obama. As FOXNews put it, “white voters are hesitant to tell pollsters they won’t vote for a black candidate.” To put it in research speak, respondents are “overstating” [sounds a lot nicer than bold -ace lying, no?] their support. The reason for the overstatement is hardly a mystery—it’s often called social desirability response set which might be described as the generalized tendency for respondents in an interview to say nice (aka desirable or politically correct) things about themselves to look good and to avoid saying negative, politically incorrect things.
Anytime we have an opportunity to say something nice about ourselves—we watch PBS, brush and floss our teeth three times a day, and have sex 10 times a week—we tend to take it. Conversely, if we have the opportunity to under-report negative or socially undesirable attitudes and behaviors—we hardly drink at all and certainly never watch the Home Shopping Network—we’ll take that too.
Pollsters know about this bias but have ignored such effects because they’re not that important in a typical poll or election. In most political races, there isn’t a socially desirable or politically correct response—there’s just candidate A, candidate B, or none of the above. They might as well be asking if you prefer Coke or Pepsi, white, wheat or rye bread. But in some races—Obama v. McCain as an example—there just might be a “right” answer respondents feel they need to give.
The real mystery here is why pollsters aren’t checking for the existence of response bias this year. By asking respondents who their best friend is going to vote for, for instance, they could gauge the level of bias present. Pollsters could also do their interviews in person in locations around the country where people can actually go into a simulated voting booth. In these cases, the prospective voters are convinced of their anonymity and are much more likely to express the truth than they are when responding to a stranger on the telephone. The latter method is more expensive to execute than conventional polling, but much more likely to yield valid answers particularly where social desirability response bias is likely to occur. Given the drubbing polls have taken in recent years for inaccuracies and lack of predictive capabilities, it may not be a bad idea for pollsters to consider.
-October 2008
This content was provided by Copernicus Marketing Consulting and Research. Visit their website at www.copernicusmarketing.com.
Other content shared by Copernicus Marketing Consulting and Research
Get More From Advertising Tracking Research
by Copernicus Marketing Consulting and Research
Advertising tracking research often returns results that don't help marketers: it can take months for tracking numbers to change significantly and even when they do, they provide little insight into why they went up or down. Here are three things you can do today to get more from advertising tracking research tomorrow. Read Article »
Psychographic Profile: What Your Position on the Legalization of Marijuana Might Say About You
by Copernicus Marketing Consulting and Research
Psychographics include variables related to attitudes, interests, personality characteristics, and values. This psychographic profile looks at a side-by-side comparison of the attitudes, behaviors and demographics of the opposite sides of a hotly debated topic: the legalization of marijuana. Read Article »
Good Versus Bad Market Segmentation
by Copernicus Marketing Consulting and Research
The qualities of good market segmentation include the results - does the research yield distinct, proprietary, highly profitable market segments? Copernicus Marketing Consulting and Research shares their 5 steps toward good market segmentation. Read Article »
Marketing Myth: Positioning Is No Longer Relevant
by Copernicus Marketing Consulting and Research
Marketers have lost control of certain aspects of brand communication - it is no longer enough to identify and communicate one brand position. Brands now target specific groups of consumers to ensure their message is relevant, compelling and motivating. One of the most important steps is using customer input, as described in this article. Read Article »
Innovation Needn't Be Such a Scary Proposition For Marketers
by Copernicus Marketing Consulting and Research
Marketers view innovation as key to combating hard economic times, yet less than half have expressed satisfaction with innovation efforts. Creating a new product includes finding the perfect price: market research helps pin down this element and in turn helps produce a market winner. Read Article »
Attitudinal and Behavioral Profile of People Who Are In Excellent Health
by Copernicus Marketing Consulting and Research
Do you start every conversation with litany of what ails you? A behavioral and attitudinal comparison of people who say they are NOT in excellent health vs. those who say they are. Read Article »
You Can Get Marketers To Eat Their Spinach
by Copernicus Marketing Consulting and Research
Marketers tend to distrust research and data, even though in principle they agree research is good for decision-making. This article shares tips for getting marketers to bring together principle and practice. Read Article »
Study Shows Emotional Connection Currently More Pipe (or Hype) Dream Than Reality
by Copernicus Marketing Consulting and Research
Brands strive to build an emotional connection with consumers, but how many consumers actually respond to these efforts? This article reveals findings from a study of U.S. consumers about emotional connection to brands. Read Article »
Marketing Decisions Still Made With More Guts Than Brains
by Copernicus Marketing Consulting and Research
This article shares findings from a survey of marketing executives and their decision-making approaches - intuitive judgment versus data and research-based. Read Article »
Interview: Adapting Marketing Research Tools for the Digital Age
by Copernicus Marketing Consulting and Research
Marketing Fray sat down with Copernicus' Eric Paquette, a senior vice president at the firm who's leading the digital charge, and posed three questions about how and what marketers can do to make good digital marketing decisions. Read Article »
3 Tools to Get a Read on Marketing Performance in 30 Days
by Copernicus Marketing Consulting and Research
Why Doesn't My Market Segmentation Work?
by Copernicus Marketing Consulting and Research
This article shares scenarios where you might run into problems with market segmentation. Read Article »
Psychographic Profile: East Coast vs. West Coast Stereotypes
by Copernicus Marketing Consulting & Research
Psychographics include variables related to attitudes, interests, personality characteristics, and values. This article shares insights on East Coast vs West Coast stereotypes. Read Article »
Top 10 Reasons for New Product Failure
by Copernicus Marketing Consulting and Research
This article presents reasons why only 10-20% of new products and services succeed. Read Article »
Rolling Out a New Product? Take 3 Steps to Nail Execution
by Copernicus Marketing Consulting and Research
While finding the right new product or service is a pretty high hurdle in and of itself, there’s also the marketing plan and the often overlooked execution plan to consider. Here are three steps you can take to keep all the parts of the marketing plan heading in the same direction as you roll it out into market. Read Article »
How to Identify the Most Profitable Targets
by Copernicus Marketing Consulting and Research
A top ten list of criteria used to identify financially-optimal targets. Read Article »
Simulated Test Marketing Gets Your New Products/Services Off on the Right Foot
by Copernicus Marketing Consulting and Research
Simulated test marketing research enables you to experiment and hone in on the best marketing plan. Learn more about this method and its application in this article. Read Article »
"Actionable Insights": Watchword or Buzzword?
by Copernicus Marketing Consulting and Research
This article discusses the measurement of the return on marketing research investments and what to with all the “insights” into customers and prospects. Read Article »
Customer Satisfaction in a Slump: Even the Net Promoters Agree
by Copernicus Marketing Consulting and Research
This article discusses the the latest statistics from the American Customer Satisfaction Index (ASCI). Read Article »
Interview With An Expert: Rolf Olsen on Social Media Listening
by Copernicus Marketing Consulting and Research
Social conversation may offer marketers a constant stream of real-time data on trends, unmet customer needs, and campaign and new product performance, but there’s a whole host of questions that marketers still have to answer when it comes to using and applying information gathered via social listening to marketing decisions. Read Article »
10 Need-to-Knows to Find Good Customer Targets in the Digital Age
by Copernicus Marketing Consulting and Research
In addition to revenue measures, there are other characteristics that make one customer more valuable than another because they’re easier to get and keep, as well as engage as co-marketers. Here are 10 need-to-knows about current and prospective customers that will clearly and definitively guide marketers to good customer targets in the digital age. Read Article »
5 Ways to Eliminate Advertising ROI Anxiety in 2012
by Copernicus Marketing Consulting and Research
Sticking with tried-and-true media vehicles that have performed reliably may be one way marketers can alleviate advertising ROI anxiety. We have five more solutions to knock it out all together in 2012. Read Article »
