White Paper:
Deriving Value from Research: the Use of Conjoint Analysis for Product Development
by Rajan Sambandam, TRC
Marketing research has been used by firms over the last several decades to provide information for decision making. Over time, increasingly sophisticated statistical methods have been developed and deployed in the service of this goal. This article focuses on one such method - conjoint analysis - and its application to product development.
Conjoint Analysis: An Overview
The origins of conjoint analysis have been traced all the way back to the mid 1960s. Since that time, tremendous advances have been made in the development of new types of conjoint analysis as well as in better estimation methods. Conjoint analysis is a decompositional technique in that entire product concepts are shown to respondents and attribute importance scores are derived from the product ratings. The types of conjoint analyses available include full profile models using ratings or rankings, hybrid conjoint models, adaptive conjoint models and discrete choice conjoint models.
The nature and complexity of the problem at hand determines the use of a particular method. For simple problems (less than ten attributes), full profile conjoint analysis with ratings or ranking could be used. Alternatively, discrete choice conjoint could also be used. In the case of more complicated problems, hybrid models or adaptive conjoint analysis could be used.
In conjoint analysis terms, products are described as bundles of attributes. A typical conjoint study entails the careful development of attributes and levels of a product. Starting a conjoint analysis study without this crucial first step is a bad idea. Simpler conjoint studies typically have less than ten attributes and approximately three to five levels per attribute. If in fact there are six attributes and three levels per attribute, the total number of product combinations possible is 729. Since all of these cannot be presented to a respondent, a fractional factorial design is often used to reduce this down to a manageable number (in the 15-25 range).
Data collection can be conducted either in person, by mail or online. For in person data collection, respondents would have to be brought to a central location. With mail-based data collection, a phone recruit is typically needed to recruit qualified respondents. These recruits are then mailed a questionnaire containing the relevant product concepts. Depending on cost and time constraints, answers can then be mailed back or collected with a follow up phone call. Online data collection also usually requires a phone recruit, unless a clean list of email addresses is available.
With more complex studies, where more than 10 attributes are used, even a fractional design can produce an unwieldy number of product concepts for respondents to rate. In such cases, adaptive conjoint analysis can be very useful. With this method respondents are asked a series of questions to identify attributes that are important to them. They are then presented with partial profiles (i.e.) incomplete product descriptions and asked to rate them. Based on the answers provided to the first few questions the later questions are customized for the particular respondent. This dynamic nature of questioning in adaptive conjoint analysis is the reason why a respondent needs a computer to enter responses. Hence, data collection options are more restricted when studies are more complex.
Product Development
Development of products (or services) can be done either from scratch or based on modifications to existing products. Analytically this task can be approached in several ways. The simplest method, especially for product modifications, is to ask direct questions about features that respondents may like or dislike. Since there is no inherent trade-off involved in this process, the importance associated with certain features may be overestimated.
An alternative method is to provide respondents with full product concepts and ask questions about the attractiveness of the product. In this method, often more than one product is presented. However, unlike in conjoint analysis, there is no design dictating the combinations to be shown and hence the ratio of information obtained for the effort expended is quite low.
Conjoint analysis is the most efficient and economical method for product development. The conjoint design ensures that even though a fraction of the total number of possible combinations are shown to each respondent, the best information can be derived from it. The process of designing the conjoint entails a disciplined thought process that is very useful well beyond the completion of the study. Next, we will look at an application of conjoint analysis to the redesign of an existing program.
Case Study
Recently, we had the opportunity to work with a California based energy utility to re-design their current load management program. Customers who enrolled in the program could have their electric power cut for specified periods when there was a power emergency in the region. For this, participating customers were provided monetary compensation. Since there was a significant drop-off in participation in the program from previous years, the utility wanted to conduct a study to identify the attributes of the program that were important to participants. The hope was that this understanding would lead to a better-designed program that would not only retain more customers, but would increase enrollment as well. Prospective respondents were recruited by phone, mailed the conjoint design and called again to obtain their responses. The five attributes included in the study were intensity of outage, monetary compensation on a daily and seasonal basis, and duration of power outage in hours per day and days per month. Levels used for each attribute are shown in the following table.
A full profile discrete choice conjoint design was used since there are so few attributes and because discrete choice is more realistic in this case. Each respondent evaluated ten tasks. Every task offered a choice between three product combinations or none at all.
Utility scores were estimated using the Hierarchical Bayes estimation procedure. Using this method, utilities can be estimated at the individual level and then aggregated to any desired level. Such a process provides better results than estimating the utilities directly at the aggregate level.
In total, 365 target consumers were included in our analysis. The importance weights (rounded to the nearest whole numbers) for each of the attributes is given in the table. These weights are calculated based on the range of utility scores obtained for each attribute.
As the results show, the intensity of the power outage (measured in 20-minute increments) was by far the most important attribute. Respondents wanted to limit discomfort and hence placed more importance on this attribute than on monetary compensation. Both compensation attributes had quite similar importance weights. This is a significant result, since fixed seasonal payments are less expensive for the company to implement than payment per day used. The number of hours per day and days per month were the least important of the attributes tested. This means that respondents were relatively unconcerned about long periods of power reduction as long as there were substantial breaks between the outages.
Interestingly, when we asked respondents directly about changes they would like to see in the program, increasing the monetary compensation was by far the most popular option. Unlike the conjoint exercise, however, specific dollar amounts were not measured in this exercise, so the results are not directly comparable. Yet, it does provide some indication of the differences in results possible if, instead of using a conjoint design, we were to simply ask respondents about what they feel is important.
Conclusions
This study was conducted before the rolling blackouts hit California earlier this year. However, there had been a considerable amount of dissatisfaction with this program over the years leading to a drop in enrollment, which necessitated this study. Results from this study have given the utility a sound basis for making desirable changes to the program to retain current customers and enroll new ones.
Due to the high level of importance given to intensity of outage in the conjoint study, the utility offered a variety of levels in its redesigned program to allow customers to choose their level of comfort. Hence, even though the conjoint study did not test every level that was ultimately offered, it quite clearly was the impetus behind such a move. The importance given to monetary compensation was translated by the utility into a dual incentive system, which used both fixed seasonal payments as well as usage based payments. Finally, usage was restricted to no more than four hours per day and sixteen days for the season barring emergencies.
This study is one example of the successful application of conjoint analysis. The many forms of conjoint analysis have been used in numerous industries over the years in the area of product development with great success. With each advance made in this area of research, the opportunities to use this unique method continue to grow.
This article was written by Rajan Sambandam of TRC, a full-service market research provider located in Fort Washington, PA.
Other content shared by TRC
Better Questions For Segmentation: Use of MAX-DIFF
by Rajan Sambandam, TRC
Using Maximum Difference Scaling as a method in designing surveys may ensure more useful results in your market research. It is a comparative method based on importance that sidesteps the problems associated with traditional importance scales. TRC explains the mechanics behind this method through a detailed example in this white paper. Read Article »
Database Scoring with Object Based Segmentation
by Rajan Sambandam, TRC
Segmentation created from company databases are often lacking the rich segmentation schemes formed by attitudinal surveys. A new approach is Object based segmentation that uses database variables at the basis for forming attitudinal segments, leaving both markets classifiable with clear demographic segments. TRC compares traditional segmentation analysis with Object based. Read Article »
Asymmetry in Product Features: Use of the Kano Method
by Rajan Sambandam, TRC
The presence or absence of product features strongly affect consumer satisfaction with the design. Comparing these features using asymmetry analysis can help identify satisfiers and dissatisfiers from among the features of a product. The Kano method is similar but results in categorizing each respondent's answers. TRC presents this essential method of deciding new product features in detail. Read Article »
Conjoint Analysis versus Self-Explicated Method: A Comparison
by Rajan Sambandam, TRC
Determining feature importance in a product can be divided into two techniques - top-down methods where a customer evaluates the whole product at once, and bottum-up methods where features are evaluated individually or in sets. The former method, Conjoint Analysis, is more common while the latter method, Self-Explicated Method, is not widely used but has practical advantages. TRC compares the two methods in this white paper. Read Article »
Product Configurator
by Rajan Sambandam, TRC
To help customers purchase the right product, companies often use product configurators - tools that let customers design their purchase before ordering. This method is employed as a market research technique, similar to conjoint analysis but without some of the constrictions. This white paper from TRC explains an appropriate use of the product configurator method. Read Article »
Market Segmentation: One Method, Four Examples
by Rajan Sambandam, TRC
Effective market segmentation requires an understanding of the market and the skilled art of finding the appropriate segments. TRC gives four examples of this method's application with results. Read Article »
How to Measure the Value of a Brand
by Rajan Sambandam, TRC
Brand name evokes an inherent value; finding a way to reliably measure that value is crucial in determining product development. A technique called discrete choice conjoint analysis is described in this paper by TRC. Read Article »
Cluster Analysis Gets Complicated
by Rajan Sambandam, TRC
Segmentation studies using cluster analysis have become commonplace. However, the data may be affected by collinearity, which can have a strong impact and affect the results of the analysis unless addressed. This article investigates what level presents a problem, why it's a problem, and how to get around it. Simulated data allows a clear demonstration of the issue without clouding it with extraneous factors. Read Article »
Identifying Feature Importance: A Comparison of Methods
by TRC
Understanding what customers want is fundamental to the new product development process as well as to the process of keeping existing products fresh and relevant. To be successful in this area we need to be able to correctly identify what features are important to consumers. Feature importance can be measured using a variety of methods of differing effectiveness. In this paper we will deal with the following methods: Importance Scales, Pick data, Pairwise Comparisons, and Max-Diff. Read Article »
Monadic Price Testing vs. Price Laddering
by TRC
Compares two popular pricing methods to understand the difference in take rate information. Read Article »
New Product Development: Stages and Methods
by Rajan Sambandam, TRC
TRC identifies the best methods for each stage of the product development process, from Idea Generation through Feature Development, Product Development and Product Testing. Read Article »
Understand Choice in Banking: Use of Discrete Choice Conjoint Analysis
by TRC
Conjoint analysis provides incentive for survey respondents to determine which features must not be omitted in their final purchase. The method closely mirrors decision-making in the real world, and as shown by TRC in this white paper, is applicable to many situations including how customers choose their bank. Read Article »
Want better product ideas? Try smart incentives
by Rajan Samandam, TRC
Idea generation from survey respondents is strongly dependent on incentive. Introducing competition strengthens the quantity and quality of creative responses. TRC provides examples of smart incentives in this white paper. Read Article »
An alternative method of reporting customer satisfaction scores
by Rajan Sambandam and George Hausser of TRC
Though customer satisfaction evaluations are widely used, reporting of these scores has varied from one study to another. This is likely the result of each method’s advantages and disadvantages, as well as the personal preferences and habits of the researcher. In this article we review various reporting methods and outline our method with an example. Read Article »
Identifying the Key Drivers of Brand Image
by TRC
Measuring brand image requires looking at direct effects as well as indirect effects of a company's performance. TRC compares traditional multiple regression with SatiscanTM, a method that can review all possible path models. Read Article »
Improving Call Satisfaction: A Case Study
by TRC
TRC presents a case study of analyzing and improving a call center as an on-going data collection process. Read Article »
Improving Claim Satisfaction: A Case Study
by TRC
A case study on applying full-service market research to help an insurance company improve their client satisfaction with claim handling. Read Article »
Non-Response Bias In Survey Sampling
by TRC
Market research accounts for many scenarios to ensure high quality of data. One of the most overlooked problems is non-response bias. TRC describes ways to reduce its effects through survey design and data adjustment in this white paper. Read Article »
Segmentation Success
by Michael Sosnowski, TRC
This paper explains the basic building blocks of the segmentation process and its implementation. Read Article »
Survey of Analysis Methods Part I
by Rajan Sambandam, TRC
Practical marketing research deals with two major problems: identifying key drivers and developing segments. In this two-part series TRC looks at key driver analysis and segmentation. Read Article »
Survey of Analysis Methods Part II
by Rajan Sambandam, TRC
This is Part II of a series looking at aspects of practical marketing research: identifying key drivers and developing segments. This content describes specific segmentation methods: cluster analysis, neural networks, self-organizing map (SOM), and mixture models. Included is a discussion on ideas for developing good segments. Read Article »
Validating Satiscan Using A Split Sample Approach
by TRC
TRC's SatiscanTM model is tested for validity using call center data and a split sample approach. This shows that SatiscanTM produces similar models when run on random halves of an energy industry dataset. Read Article »
Satiscan and Regression Analysis: A Comparison
by TRC
The comparison shows the advantages of SatiscanTM, an analytical method from TRC, over regression in identifying the correct and cost efficient action steps. Read Article »
TURF: New Methods for Implementation
by Westley Ritz, TRC
TURF is a long-established and quite useful marketing research tool, but not everyone is familiar with how it works, or with the latest developments that can make TURF even more effective. The purposes of this paper are twofold: (1) to explain the technique and (2) to describe the latest methods for implementation. Read Article »
Product Configuration: A Research Approach for the Times
by Rajan Sambandam & Pankaj Kumar, TRC
The marketplace has shifted in the last decade with the ability of consumers to configure the product they want. This white paper explains the basics of configuration, an approach that mimics the real world of customer driven product design to obtain insight into consumer decision-making. Read Article »
Product Configuration: Evidence for Effectiveness
by Rajan Sambandam & Pankaj Kumar, TRC
This white paper looks at the examples from one product configuration study, the kinds of information that can be derived and the possibilities provided by statistical analysis. Read Article »
