An Open Discussion on the Impact of Respondent Sourcing

By Frank Kelly

Research clients should learn how the source of their respondents affects research data. At Lightspeed GMI, we have been observing some very significant differences depending on the source of the data. The way a respondent enters a study will impact the overall data quality obtained from that respondent. It is not a story of good and bad, but rather just a case of significant differences; ignoring these differences could result in poor quality research.

Clients still believe the respondent world is a choice between research panel respondents and river respondents. In reality, the river sourcing technique has not existed for several years. What we have instead is a range of panel types that mostly have nothing to do with research; members of these panels need to do something in order to get something they want, in this case, complete a survey. The important factor here is that river used to indicate ‘freshness,’ which was seen as a virtue when compared to research panel respondents that were seen as ‘conditioned.’ A dynamically sourced respondent (which is what we call the process of amalgamating non-research panel respondents) actually takes many more surveys than a research panel respondent.

The majority of respondents sourced from most panel companies are actually dynamically sourced respondents that come from a variety of places (i.e., social media sites, game sites and loyalty sites).  These are often mixed with respondents sourced from traditional research panels. The key point here is that we get very different data depending on the source. Dynamic source respondents consistently rate product concepts higher, they consistently have a higher incidence of qualifying for studies (due to over-claiming behavior) and they are less attentive survey takers. We can fix attentiveness and over-claiming through our data quality process, but the difference in the way they answer questions cannot be fixed.

At Lightspeed GMI, we believe that research panel respondents are the preferred method for your research needs, but we also feel that a small amount of dynamically sourced respondents can be beneficial for most studies. We normally recommend an 85% panel/15% dynamic source blend. This blend enables us to take advantage of clear dynamic sourcing benefits: it is inexpensive and has complimentary demographics to panels (lower income, larger households, less education and more ethnic populations). If we blend at this level, we tend to stay closer to category benchmarks than at higher levels of dynamic sourcing.

Consistency is key. You should determine if you will be using research panel respondents, dynamically sourced respondents or a blend. The ratio of these sources should be help consistent for all quota cells in the study and if the study is a tracker or wave study, the blend should be consistent form period to period.