Asking Discrete Choice Questions When the Answer is, “It Depends”

Discrete choice questions are typically straightforward – “Which product do you prefer?”  However, in some situations, the respondent may be thinking, “It depends.” This article describes these situations and some possible solutions.

Identifying Issues in Discrete Choice Questions
In a conjoint/discrete choice exercise, respondents are typically asked a series of questions in which they must choose their preferred product or service among several, each with different combinations of features and pricing. The question is often straightforward, such as “Which of the following widgets do you prefer?” The respondent selects the preferred product among several offered. However, there are times when the respondent may want to say, “It depends”. For example:

• Airline travel: Business or leisure? The business traveler may find schedule more important, whereas the leisure traveler may focus more on price.
• Beverages: For what occasion? In the morning you may want something hot, whereas after a workout you may prefer something cool and refreshing.
• Hotels: For family vacation or romantic getaway? Preferences, as well as the relevant competitive set, can vary depending on the occasion.
• Prescription drugs: What type of patient? Old or young? How bad is the condition? The preferred medication will likely differ by patient type.
• Vitamins: For whom? Yourself? Your children? The ideal product will vary depending on the intended user.

While taken from different product categories, the above examples share a common characteristic. In all cases, the respondent would likely answer, “It depends!” In such cases, the respondent requires a frame of reference in order to make a meaningful choice. In fact, without a frame of reference, you really do not know what the respondent had in mind when making a choice.

Getting Better Results from a Discrete Choice Exercise
A frame of reference is simply a context for the choice the respondent is being asked to make. Our recommendation: don’t leave context up to the respondent. Not only is it difficult for the respondent to answer the choice questions, but it is equally difficult to interpret the results without knowing what context the respondent had in mind.

A frame of reference should be provided whenever a choice depends on context. There are several possible solutions to the question of context in discrete choice. Here are a few that we have used with great success:

• Narrow the focus of the study to one context: Take the most typical context, the one that represents your highest volume, or the one most closely related to the study’s objectives – and provide that context to the respondent. “Which of the following beverages would you prefer after working out?”
• Study two different contexts, but study them separately: Each context can be asked separately – either by splitting the sample or by including two conjoint exercises within the same study, with the respondent going through one first and then the other. “When traveling for business, which of the following flights do you prefer?” “When traveling for leisure, which of the following flights do you prefer?”
• Capture different contexts across respondents: The respondent is asked to think of the most recent purchase or occasion. This provides context to each individual respondent and provides a total picture of category preference across respondents. “Thinking of your most recent vacation, if the following hotels were available, which would you choose?”
• Allocate preference across the choices: The respondent is asked to allocate 100% across the choices shown instead of the more typical approach of having to choose the single most preferred product. “Thinking of the patients you see for heart disease, how would you allocate 100% across the three heart medications described below?”

Better Discrete Choice Results
When the answer to a conjoint question is likely to be, “It depends,” it is essential to provide a frame of reference to respondents or allow them to allocate their preferences across the choices. A frame of reference makes all the difference when interpreting conjoint results, because you will know what the respondent was thinking when answering the discrete choice questions. While narrowing the focus may seem like a restriction, in the end, it is better to have a narrower focus and know what the respondent was thinking than to have a broader focus and not know how to interpret the results.

Sawtooth Technologies Consulting Group helps solve complex business problems by shedding light on your market. Our team has decades of experience in applying conjoint and discrete choice analysis, MaxDiff, and other advanced quantitative research methodologies to generate insights that lead to better-informed business decisions.

-August 2010

This content was provided by Sawtooth Technologies. Visit their website at www.sawtooth.com.