Article:
New Adaptive Choice-Based Conjoint Technique Shows Promise
by Jon Godin, Director of Analytical Services, Chadwick Martin Bailey, Inc.
Adaptive Choice-Based Conjoint is a new conjoint technique that is an approach to discrete choice that would “adapt” to respondent answers as the task progresses. This article provides analysis of the process of this new method.
In case you missed it, a promising new conjoint technique called Adaptive Choice-Based Conjoint (ACBC) was introduced at the AMA’s Advanced Research Techniques (ART) Forum, held in June 2008. The ART Forum is an annual marketing science conference that brings together academics and practitioners for three-plus days of presentations and tutorials on the latest-and-greatest research methods our industry has to offer.
At the session, Bryan Orme, President of Sawtooth Software, presented the results of early testing they’ve completed on ACBC, which they have been developing for the past few years. Traditional choice-based conjoint (also known as discrete choice modeling) designs assume that respondents complete choice tasks by making compensatory trade-offs: that is, if they really want feature A, but it is not available, they will seek out a package that contains other features that will compensate for the loss of feature A. However, in practice, respondents will often resort to non-compensatory heuristics, employing various rules or shortcuts to help simplify the task. These could include such things as always selecting the lowest price regardless of other features present, always choosing a favored brand, or focusing on a particular feature that they “must have” or “must avoid.”
While these rules technically infringe upon the assumptions of the standard logit choice model, the use of Hierarchical Bayes (HB) techniques when estimating utilities still enables us to make good predictions of future market behavior based on the information provided 1. However, we’re essentially getting less information than we could or should be from many respondents. Furthermore, conjoint tasks are often viewed by respondents as being repetitive, and contain offers that can be irrelevant to their actual wants and preferences. In order to address these issues, the industry has been struggling for several years to develop an approach to discrete choice that would “adapt” to respondent answers as the task progresses.
Sawtooth Software’s latest idea focuses on getting better data from each respondent, rather than trying to develop a purely adaptive model in the traditional discrete choice format. The new ACBC approach consists of three stages:
1. First, respondents are presented a Build-Your-Own (BYO) task (like you often see at a car manufacturer’s website, where you can “Build Your Own BMW” for instance), in which attribute levels are evaluated individually and traded off against price – using the car example, you could have no moon roof for $0 additional cost, or a moon roof for an extra $1,000. The respondents can choose any combination of features they desire, subject to their own budget constraint.
2. The winning product combination from the BYO task is taken into a second stage, where 24-40 “near neighbor” concepts are constructed by changing 2-5 attributes at a time. For each concept, respondents are asked whether they would consider the concept as a possibility they might purchase, or whether they would reject it. Thus, this stage of the exercise essentially produces a “consideration set.” Throughout the process of completing these screening tasks, the program can track whether respondents seem to be always including certain features (”must haves”) or always rejecting others (”unacceptables.) After enough evidence is accumulated, respondents are asked directly about these features to confirm whether a screening rule is being used. Once confirmed, all subsequent “near neighbor” tasks that contain (or are missing) those features will essentially be thrown out and replaced by new concepts that meet the screening criteria.
3. Finally, once all of the screening tasks are completed, the selected tasks that qualified into the respondent’s consideration set are brought forward into a “choice tournament,” where concepts are shown on a screen three at a time, with the respondent being asked to choose which concept they most prefer. Winning concepts move on and losing concepts drop out until a final winner is established. Often, this winning concept is deemed “more favorable” than their original BYO choice!
All of the information learned from the three tasks is then used to estimate the respondent’s utilities using a multinomial logit model. This can take the form of standard HB estimation, or a more advanced version of HB that takes into account the fact that different degrees of respondent error are present in each of the three tasks. The utilities from either approach can then be used in a simulator, just as you would with standard choice utilities.
You may be thinking that this seems like a lot to ask of respondents, and you would be correct: ACBC, based on limited testing Sawtooth has conducted, tends to take 50% to 200% longer to complete than standard discrete choice tasks. However, qualitative feedback suggests that respondents actually enjoy the ACBC exercise more, believe that it seems more tailored to their own preferences, and indeed made them want to slow down and make more careful choices. Sawtooth describes the interview experience as attempting to mimic the actual in-store buying experience that a very patient and interested salesperson would provide.
Ah, but does it actually work better? The answer here also appears to be “yes” – early tests indicate that ACBC utilities produce higher hit rates and lower mean absolute errors than standard choice utilities when used to predict responses to holdout tasks. This seems to be especially true in situations with small samples of respondents – so it appears that we are indeed getting more information about each respondent’s preferences using this new approach.
At the time of this article, dated September 2008, only three tests to date had been conducted by Sawtooth. A select group of researchers planned beta-testing forAugust 2008. The software was planned for market launch sometime around Q2 2009.
This content was provided by Chadwick Martin Bailey, Inc. Visit their website at www.cmbinfo.com.
Other content shared by Chadwick Martin Bailey, Inc.
Measuring ROI Beyond Traditional Media
by Chadwick Martin Bailey, Inc.
Beyond print and broadcast media: how to effectively measure the impact of sponsorships, experiential marketing and word-of-mouth advocacy. Read Article »
Love It or Hate It: The NPS Approach Works
by Brant Cruz, Chadwick Martin Bailey, Inc.
Customer loyalty and your bottom line: Why NPS works. Read Article »
Gear Up, Not Down: Top 5 Keys to Success in a Slow Economy
by Anne Bailey Berman, Chadwick Martin Bailey, Inc.
Five prudent strategy points to consider for your business when the economy has slowed. Read Article »
Don’t Give Up On That Segmentation Study – Do It Smarter
by Chadwick Martin Bailey, Inc.
In uncertain economic times, segmentation of your market can be difficult. This article covers top concerns addressing strategists and provides actionable insight. Read Article »
Maximize Your Data
by Chadwick Martin Bailey, Inc.
This article urges you to find multiple uses for your internally generated data through critical thinking and advanced analytics. Chadwick Martin Bailey, Inc. offers examples for applying data in new capacities. Read Article »
The Marketing Performance Advantage
by Rich Schreuer, SVP, Chadwick Martin Bailey, Inc.; CMG Partners
Measuring marketing performance generates interest but is still in its infancy. This survey of 400 companies measured the state of marketing performance, how it is implemented, and its impact. Chadwick Martin Bailey, Inc. shares insights procured from the findings. Read Article »
Taking Your Market Research Demographics off Auto-Pilot
by Cathy Harrison, Senior Project Manager, Chadwick Martin Bailey, Inc.
Recent controversy stirred by the use of an outdated demographic term by the US Census Bureau reminds us to constantly reevaluate our surveys. This article shares tips on keeping your questionaires relevant and accurate. Read Article »
Beyond Excellent Market Research: What Clients Want
by Cathy Harrison, Senior Project Manager, Chadwick Martin Bailey, Inc.
Research providers can make client satisfaction a success with these tips from Chadwick Martin Bailey, Inc. Read Article »
Segmenting the New 'Gamer'
by Chadwick Martin Bailey, Inc.
An interview with Electronic Arts' Stephen Day on their work with Chadwick Martin Bailey in developing a strategy in the gamer marketplace. Read Article »
Measuring What Matters in Super Bowl Ads: The Results
by Jeff McKenna, Senior Consultant, Chadwick Martin Bailey, Inc.
What matters is the results - did Super Bowl ads effectively reach their target demographics? This market research looks at age and income in correlation with ad preferences to try and deduce each company's target audience. Read Article »
Mobile Surveys in Market Research
by Josh Mendelsohn, VP Marketing, Chadwick Martin Bailey, Inc.
This article covers the state of mobile market research as it is today along with practical advice for making mobile surveys work. Read Article »
Qualitative Market Research: 3 Simple Tips for Getting Better Results
by Josh Mendelsohn, VP Marketing, Chadwick Martin Bailey, Inc.
Here are three simple ways to ensure your qualitative projects pay off with truly useful results. Read quotes directly from researchers regarding feedback on the proposal, the report and industry knowledge. Read Article »
Why Brand Tracking Should Include Experiences (incl. Social Media)
by Josh Mendelsohn, VP Marketing, Chadwick Martin Bailey, Inc.
Brand tracking research should cover both the impact of the brand and the impact of experience. This article suggests tracking social media channels as touch points for managing your brand image. Read Article »
Best Practices for Social Media Research
by Chadwick Martin Bailey, Inc.
How do you make sense of all the buzz around social networking sites? This article proposes three best practices for leveraging and measuring communications with your consumers through this medium. Read Article »
Brand Tracking to Help Agencies and Clients
by Chadwick Martin Bailey, Inc.
Brand tracking studies often miss key areas of design and data collection, resulting in poor campaign results. Building effective studies involves individual customization and incorporating external factors that affect data. Develop a successful program with insights from Chadwick Martin Bailey, Inc. Read Article »
A Strategic Compass For Stanley-Bostitch
by Chadwick Martin Bailey, Inc.
This case study involves a brand research project with Stanley-Bostich during economic downturn. Chadwick Martin Bailey, Inc. teamed up to discover insights about their website users, the competitive landscape and how to increase their retail shelf space. Read Article »
Market Research Shows Moving in Together Leads to Clutter
by Kristen Garvey, Director of Communications, Chadwick Martin Bailey, Inc.
Research on consumers finds that a large number of Americans find they have duplicate items when they move in with a significant other. What happens to all these items? This article looks into ways to profit from your household duplicates. Read Article »
Engaging Consumers via Social Media Makes Them More Likely To Buy, Recommend
by Chadwick Martin Bailey, Inc.
New research shows consumers engaged through social media such as Facebook and Twitter are over 50% more likely to buy and recommend than before they were engaged. This study shares both quantitative and qualitative data from a study of 1500 consumers. Read Article »
3 Steps Towards Improving Data and Processes
by Josh Mendelsohn, VP Marketing, Chadwick Martin Bailey, Inc.
One of the top challenges facing marketers is a perceived lack of the right data or processes to make informed key decisions. In addition, there are challenges in using that data for decision-making. This article from Chadwick Martin Bailey, Inc. addresses ways to resolve these issues. Read Article »
Why Measurement Alone Will Not Lead to Better Marketing
by Karl Gustafson; Rich Schreuer, Senior VP, Chadwick Martin Bailey, Inc.
A study of 400 c-level professionals was conducted by Chadwick Martin Bailey to determine how companies are tracking marketing performance. The organizations that measure and manage effectively have common success factors; read this article for practices that will improve your marketing-performance initiatives. Read Article »
Bridging the Gap Between Data and Creativity Through Market Research
by Jeff McKenna, Senior Consultant, Chadwick Martin Bailey, Inc.
The gap between by-the-numbers strategic consultants and creative ad agencies can be bridged by effective market research. In this article Jeff McKenna explains how market segmentation finds consumer preferences, needs and brand attitudes that can help influence the "emotional" and creative side of advertising. Read Article »
Qualitative Research: How the Art of Listening Has Changed
by Josh Mendelsohn, Chadwick Martin Bailey, Inc.
This article takes a look at how qualitative research has changed since the rise of social media and advances in technology. Read Article »
Just Starting Your Holiday Shopping in December? You're Not Alone
by Josh Mendelsohn, Chadwick Martin Bailey, Inc.
This content provides statistics on when consumers start their holiday shopping. If you wait until December, you should know you're not alone! Read Article »
Diet Programs Out, Interactive Fitness Products In
by Chadwick Martin Bailey, Inc.
Adults looking to lose weight have found a new method - interactive fitness products. More Americans plan on using a product such as Wii Fit or EA SPORTS Active rather than using a traditional fitness equipment or diet program. Learn more in this article from Chadwick Martin Bailey, Inc. Read Article »
Why Domino’s new ads are good for the market research industry
by Josh Mendelsohn, Chadwick Martin Bailey, Inc.
The new Domino's ads break down negative conceptions about the research industry. This article describes how the content of Domino's ads are good for the market research industry. Read Article »
Addressing Concerns Over Market Segmentation in Uncertain Times
by Josh Mendelsohn, Chadwick Martin Bailey, Inc.
This article addresses major concerns about conducting segmentation studies in the current economic situation and how to overcome these challenges. Read Article »
A Look At Purchase Decisions: Life Insurance
by Dan Gersten, Chadwick Martin Bailey, Inc.
CMB Research looks at who buys life insurance, for what reasons, and at what point in their life this purchase decision is made. The study of 1500 consumers finds that it's about more than socio-economics and life-stage events. Read Article »
Interview: Avoiding the "Brand Death" Spiral
by Chadwick Martin Bailey, Inc.
When marketing doesn't communicate with operations, the brand image is affected. Rich Schreuer provides an example of customer service not meeting brand image and how this contributes to "the brand death spiral." Read Article »
Market Research Shows IT Service Providers Lack Brand Distinction
by Don Ryan, Chadwick Martin Bailey
IT decision makers around the United States were surveyed about their use of IT-related services. The questions regarding the uniqueness of brands raise concerns about brand distinction in the industry. Read Article »
Using Primary Market Research to Evaluate B2B Social Media Strategies
by Chris Neal, Chadwick Martin Bailey, Inc.
Recent research on social media has found that engaged consumers are more likely to buy and recommend a brand. Chadwick Martin Bailey, Inc. describes this research and their recent work with a leading processor company in re-evaluating their social media effectiveness. Read Article »
Don't Underestimate the Impact of Online Guest Reviews
by Judy Melanson, Chadwick Martin Bailey, Inc.
A recent study asked consumers about social media use and attitudes as it relates to travel, finding an extremely high importance rate on guest reviews. This article shares the impact of positive and negative guest reviews on booking cruises. Read Article »
How Geo-Targeting is Affecting Customer Loyalty Programs
by Josh Mendelsohn, Chadwick Martin Bailey, Inc.
This article shares examples of how companies are engaging customers via mobile, geo-based applications such as Foursquare. Read Article »
Selling Life Insurance to the Under-30 Crowd
by Dan Gersten, Chadwick Martin Bailey, Inc.
Life Insurance: The investment product that protects more than just one's money. This article shares insights from a study of 1,500 U.S. consumers on insurance. Read Article »
Do you care about your customers? Really care?
by Judy Melanson, Chadwick Martin Bailey
If you work for a company like most, a significant portion of your research budget is dedicated to a customer feedback, performance, or satisfaction program. Are you getting the return (in insight) you are making for the investment you’re making? Read Article »
The Facts Marketers Need to Know Before Using QR Codes
by Chadwick Martin Bailey, Inc.
We set out to ask consumers what they think about the 2D bar codes known as Quick Response or QR codes. Here's what stood out, from a marketing perspective. Read Article »
