Deploying store intercepts or on-site surveys to test new concepts is often a key step in new product or service development. If not done right, it can produce useless or (even worse) misleading results. Here are the most common pitfalls associated with this type of research and how to avoid them.
 Inadequate concept specification.
Problem: Researchers often use store intercepts or on-site surveys to test ideas which are not fully specified. When this happens, respondents have trouble answering purchase intent or other overall interest questions. Alternatively, respondents may interpret what’s meant to be the same concept in many different ways, so their answers aren’t comparable.
Solution: Think like a respondent – what would they need to know about the concept in order to make a simulated purchase decision? If you (or your client) can’t provide this information, then the concept may not be ready for in-store or on-site testing.
 Missing or bad pricing.
Problem: A special case of inadequate specification is missing or bad pricing. Using a store intercept or an on-site survey to ask purchase intent when price is unknown isn’t really meaningful, while asking purchase intent when price is unrealistically high or low yields distorted results.
Solution: Conduct some form of pricing research if you (or your client) can’t provide reasonable prices. Alternatively, use the on-site concept test for other purposes (prioritizing a list of options, assessing characteristics or perceptions, etc.).
 Surveying the wrong population.
Problem: Most new products or services are intended for a certain target population. If your sample is too broadly defined, your store intercept or on-site survey results will be distorted by the inclusion of opinions from people who you’re not interested in. If your sample is too narrowly defined, your store intercept or on-site survey results will be distorted by the exclusion of opinions from people who you are (or should be) interested in.
Solution: Take the time up front to define your target audience in terms of demographics, category behavior, or other characteristics. This definition can then be implemented in the store intercept or on-site survey through sample specifications and/or screening questions. Quotas or weighting may also be needed.
 Ignoring key drivers.
Problem: The main objective of most store intercept or on-site concept tests is to assess purchase intent. To save money and minimize respondent burden, many researchers stop there. However, this information is of limited value if you don’t know why intent is high or low. In fact, understanding a concept’s strengths and weaknesses is sometimes more useful than estimating overall interest.
Solution: Include questions in the store intercept or on-site survey about concept characteristics. These may be broad or specific, and may be functional or emotional. Then use an appropriate analytical technique to determine the relative importance (impact on overall interest) of these characteristics.
 Wrong methodology.
Problem: Many researchers select their data collection method for concept tests based solely on cost and timing considerations, which usually favor the Internet. However, this may not be the right choice. For example, some target audiences (very young, very old, low income or limited education, etc.) may be hard to reach online or uncomfortable with online surveys. In such cases especially, an on-site survey or intercept based approach might be better suited.
Solution: Try to choose the methodology which yields the most valid and reliable results, even if it isn’t the fastest or cheapest. Saving time and/or money doesn’t do you any good if it comes at the expense of data quality.
 Inadequate sample sizes.
Problem: Often when deploying store intercepts or on-site surveys, cost considerations force many researchers to go with the smallest sample size that will provide a reasonable level of precision at the aggregate level. However, small samples often won’t support potentially important breakouts by key respondent characteristics. You may also want to limit some questions to certain subsamples, such as people with a minimum level of interest in the concept, and here again small base sizes can be problematic.
Solution: Focus on the smallest subsample for which you want to be able to make statistically meaningful inferences, and let that drive your total sample size. You can sometimes justify the cost of a larger sample by adding questions which address other related business issues, or alternatively by including your concept test as part of another survey.
Interested in learning more? Call us at 1-800-549-7170 or send an email to email@example.com for a free 30-minute consultation on this topic. Gold Research Inc. has extensive experience in deploying store intercepts and on-site surveys for concept testing, and we would love to talk with you about it!
- Retail Industry - General