The Dawning of a New Analytic Golden Age

By Paul Richard McCullough

For some time now, we in the marketing sciences/methodologists community, both academic and practitioner alike, have seemed to focus fairly exclusively on developing better analytic techniques.  We study in minute detail such obscure concepts as HB models with covariates and/or tuned priors, latent class choice models with C-factors and/or scale factors, finite mixture SEMs, variants on random forests, etc.   A paper I presented, for example, at the 2009 Sawtooth Software Conference compared several HB models and several latent class models and found them all to perform fairly similarly.  HB with tuned priors did better than vanilla HB, but not by much.  I’ve seen similar papers where a great deal of effort is made that only yielded small (or no) gains in model performance.  Managerial implications are typically unchanged despite all the intellectual gymnastics.

Perhaps we are squeezing all of the juice out of this lemon that there is to squeeze.  Maybe we need to look for a new lemon.

We are moving into an age of mobile devices that will allow us to survey people immediately as they experience the market environment, not days or weeks (or months or years) afterward.  Imagine conducting a conjoint exercise on the subject of laptops with respondents who have, within the past few seconds, just left a retailer that sells laptops. Or using maxdiff to measure customer satisfaction of new car buyers right after they’ve left a car dealership. Potentially very powerful stuff.

Cost per interview has plummeted and our access to very large samples continues to expand.  Samples in the thousands are now not just feasible but are also increasingly common.  We can supplement survey data with “Big Data” that take zero respondent time to collect and, in some ways, may provide more reliable information about the respondent.  We have more respondents from which to build data sets and we know more about each of them than ever before.  The potential quality of the data sets that we analyze has never been so rich.

This radically changing environment may provide opportunities to improve our research methods by huge leaps but it may require a completely different focus, a thinking well outside the current box.

Maybe we need to shift our focus from squeezing diminishing marginal improvements in statistical algorithms to searching for new ways to leverage this brave new world of expanded information.  Perhaps given these new data types and scale, there are ways to modify existing analytic techniques so that they can be extended to new platforms and better take advantage of these richer data.  Perhaps these new data will lead us to totally new analytic approaches.  Regardless of the eventual outcomes, it is an exciting time to be a methodologist.