Zombie Marketing Research: the Approach to Market Segmentation That Will Not Die

There a three major flaws of segmentation modeling that is practiced by many current marketing researchers. These flaws have been discussed and exposed over and over, and yet some research companies are still pushing these approaches to the detriment of delivering to client flawless results.

There is an over-reliance on rating scales to derive segments.

Rating scales, whether they are 1-5 or 1-7 or 0-10 or whatever, are notoriously prone to bias in their use by survey respondents. It’s well-known that some people are yea-sayers who evidence high levels of agreement, no matter the question, just as there are nay-sayers and middle-of-the-roaders.

In addition, much research has pointed out that people in different countries use rating scales differently. These tendencies make it difficult to separate true differences from scale use bias.

Analyzing data in silos discards important information.

Segmentation studies often have multiple purposes: communication, targeting, and product development, for example. By focusing the analysis on only one set of information –say attitudes, or benefits, or needs –additional potentially relevant information is overlooked.

We advocate in favor of analysis tools which are multi-dimensional in nature, taking into account, attitudes, behaviors, and personal characteristics –all within the context of the same segmentation model, rather than in silos.

Factor Analysis followed by Cluster Analysis should be totally abandoned.

FA and CA are typically combined in one analytic segmentation model, yet they define a two-stage procedure with differing objectives. FA groups questions/items into “ideas” or dimensions, while CA groups people with similar mindsets.

The two procedures have their own flaws and by combining them, the analyst may be misled by their findings. And certainly many two-stage procedures find that the flaws and errors in the first stage get carried through to the second stage.

Academics call this two-stage procedure: Tandem Clustering.

Since my statement, “Factor Analysis followed by Cluster Analysis should be totally abandoned,” is likely controversial and scary for some of you, let’s examine this a bit further.

A Critique of Factor Analysis for Segmentation

Marketing researchers often use Factor Analysis to pre-process their data before employing Cluster Analysis. Factor Analysis is applied to a battery of survey items to reduce their number and to extract meaningful dimensions or ideas which are thought to underlie the original items.

Using Factor Analysis, the analysts will often extract 60%-70% of the variability in the data, yet will “throw out” the remaining variability. Are surveys so information-rich that we can afford to discard that much?

Survey items that do not ‘load’ on any Factor are discarded. Are these items unimportant? Is it possible that these pieces of information are really the ones which discriminate across segments? Perhaps they are the items that can help us identify niche segments?

The identification of one set of factors assumes that the derived dimensions apply to everyone. In other words, everyone thinks the same. And yet, isn’t the intent of segmentation to identify differences across people? Why assume everyone is alike when differences are what you are after?

Researchers then take the identified dimensions and create factor scores for each person. Do you think that the segments you have uncovered using this transformed “factor space” truly exist if you had used the original variables? Many studies say, “No way.”

While many researchers extract factors with ratings data, we have also seen factor analysis used with binary data (applies/does not apply; describes/does not describe). Unless the analyst uses methods designed specifically for binary data, a serious problem arises.

Factor analysis is based on correlations, which range from -1.0 to +1.0. But with binary data, standard correlations will reach these maximum values only in very special circumstances. Otherwise, the correlations are artificially low (called attenuated), making the identification of the true factor structure difficult if not impossible.

Critique of Cluster Analysis for Segmentation

Cluster Analysis is a set of grouping algorithms based on “distances” between the units of analysis (people), with few true statistical tests for the goodness-of-fit of a solution.

Being based on distances limits the types of data which can be used in Cluster Analysis. If the analyst has nominal (male/female) or ordinal (low/medium/high) data, distances cannot be computed from these levels of measurement. Despite this, too many analysts blindly use Cluster Analysis on nominal, binary, or ordinal measures.

Furthermore, there are many clustering algorithms, each with its own set of rules and objectives. Studies of the performance of Cluster Analysis using synthetic data (data created where the data properties and the correct solution is known) demonstrate that some Cluster Analysis methods are much better than others in discovering the correct solution, depending upon the data you have at hand.

And different cluster methods tend to produce clusters with different properties: some look like circles, others look elongated, some with big and small clusters, others with clusters of the same size.

“Although the strategy of clustering may be structure-seeking, its operation is one that is structure-imposing.” Aldenderfer & Blashfield (1984).

This quote is saying that the method you choose (and its underlying result tendencies) will highly influence what you find. If you don’t pay attention to the assumptions behind the tools that you use, be prepared to find something you may not like. Or, worse yet, be prepared to discover something that really isn’t there. With so many clustering algorithms to choose from – and each one capable of delivering a different answer from your data – which one should you use?

Cluster Analysis applies no “penalty” for extracting too many segments. It’s too easy to over fit a cluster solution. And, with Cluster Analysis, people get assigned to a segment with certainty. Shouldn’t we acknowledge that our models and methods are not perfect and hence quantify the uncertainty inherent in our results?

Finally, the most popular clustering algorithm among marketing researchers – K-Means – is saddled with several drawbacks. It cannot use nominal, binary, or ordinal data. It can be sensitive to order of the data in the dataset. It assumes the variability of each of the items used to cluster are the same.

It is sensitive to outliers. There are no good statistical tests for the “correct” number of clusters to retain. And K-Means tends to find clusters that are of the same size. In other words, small niche segments are very hard to discover with K-Means.

ZOMBIE MARKETING RESEARCH

“Tandem clustering is an out-moded and statistically insupportable practice.” Arabie & Hubert (1994).

I will never totally understand how Tandem Clustering still remains viable in marketing research and in other sciences. The drawbacks are too many for me to count. But if you insist, go right ahead.

References 

Aldenderfer, M. S., & Blashfield, R. K. (1984). Cluster analysis. Beverly Hills, CA: Sage Publications.

Arabie, P. and Hubert L.J. (1994). Cluster Analysis in Marketing Research. In Advanced methods in marketing research. Ed. R.P. Bagozzi. Blackwell: Oxford, 160-189.