Service:
Validating Satiscan Using A Split Sample Approach
by TRC
TRC's SatiscanTM model is tested for validity using call center data and a split sample approach. This shows that SatiscanTM produces similar models when run on random halves of an energy industry dataset.
A common question asked of any analytical method is whether the results produced can be validated. What this means is whether the process by which the results are arrived at is consistent enough that the same results will be produced in similar circumstances. The usual method for testing the validity of a method is to split a given sample into two random parts and run the same analysis on both parts, to see if similar results can be obtained. If the results are similar in both parts of the data, then the method is considered to be valid.
This “split sample” approach is taken here to demonstrate the validity of SatiscanTM. Call center data from the energy industry was used for this purpose.
Energy Example
The dataset used here had a total of 1162 observations on 18 variables. Two random halves were created with 576 and 586 observations. Frequencies and correlation coefficients were compared to ensure that the two halves were indeed split randomly. Next, two stages of analyses were run. First, SatiscanTM analyses were run on both halves of the data. Second, stepwise regression analyses were run on both halves of the data. The regression analyses provide a benchmark for comparison, as the degree of similarity between the two regression models can be used for comparing the degree of similarity of the two SatiscanTM models.
The SatiscanTM models are given next in Figures 1 and 2 followed by the total effects tables in Table 1. Total effects tables are necessary when using SatiscanTM models since some variables can have both direct and indirect impact on the dependent variable. Total effects are calculated as the sum of direct and indirect effects.
As can be seen from Figures 1 and 2, the two SatiscanTM models are quite similar but are not exact copies of each other. The first model has more direct drivers of the dependent variable (six) than the second model (three). Further the first model also has two more key drivers (15 to 13) than the second model.
When we examine the total effects table, the most striking result is the first two key drivers. They have almost exactly the same weights in both models. The top six variables in both models are the same, although there are some differences in the ordering of the variables.
Thus in comparing the two models we could say that even though there are some differences, the basic results from both models are quite similar. Most of the dissimilar results appear in the secondary key drivers.
Next, lets take a look at the regression models presented in Table 2. In comparing the regression results with those from SatiscanTM , the most obvious difference is in the richness of the models. Because SatiscanTM is able to identify relationships between independent variables, its models have much more detail than the regression models.
Further as it was in the case of the SatiscanTM models, the first regression model has more key drivers (8) than the second model (5). Again, the variable with the most impact is the same in both cases and has almost identical impact. There is however, less commonality between the two sets of key drivers as compared to the SatiscanTM key drivers. This could possibly be because of some amount of collinearity in the data, since collinearity tends to affect regression and not SatiscanTM.
By analyzing the regression results we are able to determine that there are some inherent differences between the two samples that occurred as a result of the random split. In practice, it is often the case that when datsets are split into random halves, even though the frequencies and bivariate correlations may appear quite similar, some differences would appear in multivariate analyses. Regression analyses on the two samples helps to understand the differences between the two samples.
Taking these inherent differences into account, we could say that the SatiscanTM models are more similar than may have appeared at first sight.
Figure 1
Figure 2
This content was provided by TRC. Visit their website at www.trchome.com.
Other content shared by TRC
Better Questions For Segmentation: Use of MAX-DIFF
by Rajan Sambandam, TRC
Using Maximum Difference Scaling as a method in designing surveys may ensure more useful results in your market research. It is a comparative method based on importance that sidesteps the problems associated with traditional importance scales. TRC explains the mechanics behind this method through a detailed example in this white paper. Read Article »
Database Scoring with Object Based Segmentation
by Rajan Sambandam, TRC
Segmentation created from company databases are often lacking the rich segmentation schemes formed by attitudinal surveys. A new approach is Object based segmentation that uses database variables at the basis for forming attitudinal segments, leaving both markets classifiable with clear demographic segments. TRC compares traditional segmentation analysis with Object based. Read Article »
Asymmetry in Product Features: Use of the Kano Method
by Rajan Sambandam, TRC
The presence or absence of product features strongly affect consumer satisfaction with the design. Comparing these features using asymmetry analysis can help identify satisfiers and dissatisfiers from among the features of a product. The Kano method is similar but results in categorizing each respondent's answers. TRC presents this essential method of deciding new product features in detail. Read Article »
Conjoint Analysis versus Self-Explicated Method: A Comparison
by Rajan Sambandam, TRC
Determining feature importance in a product can be divided into two techniques - top-down methods where a customer evaluates the whole product at once, and bottum-up methods where features are evaluated individually or in sets. The former method, Conjoint Analysis, is more common while the latter method, Self-Explicated Method, is not widely used but has practical advantages. TRC compares the two methods in this white paper. Read Article »
Product Configurator
by Rajan Sambandam, TRC
To help customers purchase the right product, companies often use product configurators - tools that let customers design their purchase before ordering. This method is employed as a market research technique, similar to conjoint analysis but without some of the constrictions. This white paper from TRC explains an appropriate use of the product configurator method. Read Article »
Market Segmentation: One Method, Four Examples
by Rajan Sambandam, TRC
Effective market segmentation requires an understanding of the market and the skilled art of finding the appropriate segments. TRC gives four examples of this method's application with results. Read Article »
How to Measure the Value of a Brand
by Rajan Sambandam, TRC
Brand name evokes an inherent value; finding a way to reliably measure that value is crucial in determining product development. A technique called discrete choice conjoint analysis is described in this paper by TRC. Read Article »
Deriving Value from Research: the Use of Conjoint Analysis for Product Development
by Rajan Sambandam, TRC
Marketing research has been used by firms over the last several decades to provide information for decision making. Over time, increasingly sophisticated statistical methods have been developed and deployed in the service of this goal. This article focuses on one such method - conjoint analysis - and its application to product development. Read Article »
Cluster Analysis Gets Complicated
by Rajan Sambandam, TRC
Segmentation studies using cluster analysis have become commonplace. However, the data may be affected by collinearity, which can have a strong impact and affect the results of the analysis unless addressed. This article investigates what level presents a problem, why it's a problem, and how to get around it. Simulated data allows a clear demonstration of the issue without clouding it with extraneous factors. Read Article »
Identifying Feature Importance: A Comparison of Methods
by TRC
Understanding what customers want is fundamental to the new product development process as well as to the process of keeping existing products fresh and relevant. To be successful in this area we need to be able to correctly identify what features are important to consumers. Feature importance can be measured using a variety of methods of differing effectiveness. In this paper we will deal with the following methods: Importance Scales, Pick data, Pairwise Comparisons, and Max-Diff. Read Article »
Monadic Price Testing vs. Price Laddering
by TRC
Compares two popular pricing methods to understand the difference in take rate information. Read Article »
New Product Development: Stages and Methods
by Rajan Sambandam, TRC
TRC identifies the best methods for each stage of the product development process, from Idea Generation through Feature Development, Product Development and Product Testing. Read Article »
Understand Choice in Banking: Use of Discrete Choice Conjoint Analysis
by TRC
Conjoint analysis provides incentive for survey respondents to determine which features must not be omitted in their final purchase. The method closely mirrors decision-making in the real world, and as shown by TRC in this white paper, is applicable to many situations including how customers choose their bank. Read Article »
Want better product ideas? Try smart incentives
by Rajan Samandam, TRC
Idea generation from survey respondents is strongly dependent on incentive. Introducing competition strengthens the quantity and quality of creative responses. TRC provides examples of smart incentives in this white paper. Read Article »
An alternative method of reporting customer satisfaction scores
by Rajan Sambandam and George Hausser of TRC
Though customer satisfaction evaluations are widely used, reporting of these scores has varied from one study to another. This is likely the result of each method’s advantages and disadvantages, as well as the personal preferences and habits of the researcher. In this article we review various reporting methods and outline our method with an example. Read Article »
Identifying the Key Drivers of Brand Image
by TRC
Measuring brand image requires looking at direct effects as well as indirect effects of a company's performance. TRC compares traditional multiple regression with SatiscanTM, a method that can review all possible path models. Read Article »
Improving Call Satisfaction: A Case Study
by TRC
TRC presents a case study of analyzing and improving a call center as an on-going data collection process. Read Article »
Improving Claim Satisfaction: A Case Study
by TRC
A case study on applying full-service market research to help an insurance company improve their client satisfaction with claim handling. Read Article »
Non-Response Bias In Survey Sampling
by TRC
Market research accounts for many scenarios to ensure high quality of data. One of the most overlooked problems is non-response bias. TRC describes ways to reduce its effects through survey design and data adjustment in this white paper. Read Article »
Segmentation Success
by Michael Sosnowski, TRC
This paper explains the basic building blocks of the segmentation process and its implementation. Read Article »
Survey of Analysis Methods Part I
by Rajan Sambandam, TRC
Practical marketing research deals with two major problems: identifying key drivers and developing segments. In this two-part series TRC looks at key driver analysis and segmentation. Read Article »
Survey of Analysis Methods Part II
by Rajan Sambandam, TRC
This is Part II of a series looking at aspects of practical marketing research: identifying key drivers and developing segments. This content describes specific segmentation methods: cluster analysis, neural networks, self-organizing map (SOM), and mixture models. Included is a discussion on ideas for developing good segments. Read Article »
Satiscan and Regression Analysis: A Comparison
by TRC
The comparison shows the advantages of SatiscanTM, an analytical method from TRC, over regression in identifying the correct and cost efficient action steps. Read Article »
TURF: New Methods for Implementation
by Westley Ritz, TRC
TURF is a long-established and quite useful marketing research tool, but not everyone is familiar with how it works, or with the latest developments that can make TURF even more effective. The purposes of this paper are twofold: (1) to explain the technique and (2) to describe the latest methods for implementation. Read Article »
Product Configuration: A Research Approach for the Times
by Rajan Sambandam & Pankaj Kumar, TRC
The marketplace has shifted in the last decade with the ability of consumers to configure the product they want. This white paper explains the basics of configuration, an approach that mimics the real world of customer driven product design to obtain insight into consumer decision-making. Read Article »
Product Configuration: Evidence for Effectiveness
by Rajan Sambandam & Pankaj Kumar, TRC
This white paper looks at the examples from one product configuration study, the kinds of information that can be derived and the possibilities provided by statistical analysis. Read Article »
