Deriving Value from Research: the Use of Conjoint Analysis for Product Development

Rajan Sambandam, TRC

Marketing research has been used by firms over the last several decades to provide information for decision making. Over time, increasingly sophisticated statistical methods have been developed and deployed in the service of this goal. This article focuses on one such method - conjoint analysis - and its application to product development. We will briefly look at what conjoint analysis is and a real life example of its application that provided true value to a company.

Conjoint Analysis: An Overview
The origins of conjoint analysis have been traced all the way back to the mid 1960s. Since that time, tremendous advances have been made in the development of new types of conjoint analysis as well as in better estimation methods. Conjoint analysis is a decompositional technique in that entire product concepts are shown to respondents and attribute importance scores are derived from the product ratings. The types of conjoint analyses available include full profile models using ratings or rankings, hybrid conjoint models, adaptive conjoint models and discrete choice conjoint models.

The nature and complexity of the problem at hand determines the use of a particular method. For simple problems (less than ten attributes), full profile conjoint analysis with ratings or ranking could be used. Alternatively, discrete choice conjoint could also be used. In the case of more complicated problems, hybrid models or adaptive conjoint analysis could be used.

In conjoint analysis terms, products are described as bundles of attributes. A typical conjoint study entails the careful development of attributes and levels of a product. Starting a conjoint analysis study without this crucial first step is a bad idea. Simpler conjoint studies typically have less than ten attributes and approximately three to five levels per attribute. If in fact there are six attributes and three levels per attribute, the total number of product combinations possible is 729. Since all of these cannot be presented to a respondent, a fractional factorial design is often used to reduce this down to a manageable number (in the 15-25 range).

Data collection can be conducted either in person, by mail or online. For in person data collection, respondents would have to be brought to a central location. With mail-based data collection, a phone recruit is typically needed to recruit qualified respondents. These recruits are then mailed a questionnaire containing the relevant product concepts. Depending on cost and time constraints, answers can then be mailed back or collected with a follow up phone call. Online data collection also usually requires a phone recruit, unless a clean list of email addresses is available.

With more complex studies, where more than 10 attributes are used, even a fractional design can produce an unwieldy number of product concepts for respondents to rate. In such cases, adaptive conjoint analysis can be very useful. With this method respondents are asked a series of questions to identify attributes that are important to them. They are then presented with partial profiles (i.e.) incomplete product descriptions and asked to rate them. Based on the answers provided to the first few questions the later questions are customized for the particular respondent. This dynamic nature of questioning in adaptive conjoint analysis is the reason why a respondent needs a computer to enter responses. Hence, data collection options are more restricted when studies are more complex.

Product Development
Development of products (or services) can be done either from scratch or based on modifications to existing products. Analytically this task can be approached in several ways. The simplest method, especially for product modifications, is to ask direct questions about features that respondents may like or dislike. Since there is no inherent trade-off involved in this process, the importance associated with certain features may be overestimated.

An alternative method is to provide respondents with full product concepts and ask questions about the attractiveness of the product. In this method, often more than one product is presented. However, unlike in conjoint analysis, there is no design dictating the combinations to be shown and hence the ratio of information obtained for the effort expended is quite low.

Conjoint analysis is the most efficient and economical method for product development. The conjoint design ensures that even though a fraction of the total number of possible combinations are shown to each respondent, the best information can be derived from it. The process of designing the conjoint entails a disciplined thought process that is very useful well beyond the completion of the study. Next, we will look at an application of conjoint analysis to the redesign of an existing program.

Case Study
Recently, we had the opportunity to work with a California based energy utility to re-design their current load management program. Customers who enrolled in the program could have their electric power cut for specified periods when there was a power emergency in the region. For this, participating customers were provided monetary compensation. Since there was a significant drop-off in participation in the program from previous years, the utility wanted to conduct a study to identify the attributes of the program that were important to participants. The hope was that this understanding would lead to a better-designed program that would not only retain more customers, but would increase enrollment as well. Prospective respondents were recruited by phone, mailed the conjoint design and called again to obtain their responses. The five attributes included in the study were intensity of outage, monetary compensation on a daily and seasonal basis, and duration of power outage in hours per day and days per month. Levels used for each attribute are shown in the following table.

A full profile discrete choice conjoint design was used since there are so few attributes and because discrete choice is more realistic in this case. Each respondent evaluated ten tasks. Every task offered a choice between three product combinations or none at all.

Utility scores were estimated using the Hierarchical Bayes estimation procedure. Using this method, utilities can be estimated at the individual level and then aggregated to any desired level. Such a process provides better results than estimating the utilities directly at the aggregate level.

In total, 365 target consumers were included in our analysis. The importance weights (rounded to the nearest whole numbers) for each of the attributes is given in the table. These weights are calculated based on the range of utility scores obtained for each attribute.

As the results show, the intensity of the power outage (measured in 20-minute increments) was by far the most important attribute. Respondents wanted to limit discomfort and hence placed more importance on this attribute than on monetary compensation. Both compensation attributes had quite similar importance weights. This is a significant result, since fixed seasonal payments are less expensive for the company to implement than payment per day used. The number of hours per day and days per month were the least important of the attributes tested. This means that respondents were relatively unconcerned about long periods of power reduction as long as there were substantial breaks between the outages.

Interestingly, when we asked respondents directly about changes they would like to see in the program, increasing the monetary compensation was by far the most popular option. Unlike the conjoint exercise, however, specific dollar amounts were not measured in this exercise, so the results are not directly comparable. Yet, it does provide some indication of the differences in results possible if, instead of using a conjoint design, we were to simply ask respondents about what they feel is important.

Conclusions
This study was conducted before the rolling blackouts hit California earlier this year. However, there had been a considerable amount of dissatisfaction with this program over the years leading to a drop in enrollment, which necessitated this study. Results from this study have given the utility a sound basis for making desirable changes to the program to retain current customers and enroll new ones.

Due to the high level of importance given to intensity of outage in the conjoint study, the utility offered a variety of levels in its redesigned program to allow customers to choose their level of comfort. Hence, even though the conjoint study did not test every level that was ultimately offered, it quite clearly was the impetus behind such a move. The importance given to monetary compensation was translated by the utility into a dual incentive system, which used both fixed seasonal payments as well as usage based payments. Finally, usage was restricted to no more than four hours per day and sixteen days for the season barring emergencies.

This study is one example of the successful application of conjoint analysis. The many forms of conjoint analysis have been used in numerous industries over the years in the area of product development with great success. With each advance made in this area of research, the opportunities to use this unique method continue to grow.

This article was written by Rajan Sambandam of TRC, a full-service market research provider located in Fort Washington, PA.

[Nov 10, 2009]