Online Market Research Surveys - Survey Length

Marc I. Tillman, Amplitude Research, Inc.

One of the more important aspects of designing online market research surveys is the length of the survey. This should be discussed at the beginning of the survey process as the time it takes for respondents to complete the survey can impact sample costs and the fee for survey hosting services.

Many online market research firms adopt a general policy of three survey questions being equivalent to one-minute of online market research. However, in the field, the timing can be far different depending on the number of words in the survey, the "thinking time" required by respondents, the manner in which the questions are programmed, and the speed of the survey technology. For example, a question formatted as a radio table with column headings consisting of a 5-point scale and simple rows of questions, containing several words each, can equate to 10 questions a minute. On the other hand, message or image displays with lengthy statements or open-ended comment questions can equate to only one question per minute.

The professional staff at most online market research firms should be able to estimate the length prior to survey launch. Online survey companies that offer respondent timers can use those timers during the survey preview period to estimate response times while also using the timers in the field. Keep in mind for cost purposes that the median response time is not necessarily the statistic used for sample pricing. For example, if the median survey completion time is 15 minutes, but 25% of the respondents take 22 minutes or longer, those respondents will expect to receive a survey incentive commensurate with their response times. This could increase the CPI for the entire project.

As a general rule, longer survey length in online market research is more expensive in B2B market research than in consumer market research - - although there are exceptions (such as high net worth households). Keeping B2B market research at 10 minutes or less can often provide a big assist in limiting sample costs. Survey length of 20 minutes or more can dramatically increase costs, with sample incentives rising significantly with each 5-minute increase.

Clients should also keep in mind the validity of the survey data as survey length exceeds 20 minutes. It might not be possible to ask every question in a single survey instrument. Survey respondents, like everyone else, can get distracted or become tired after a while and their responses can be less thoughtful.

In summary, the time it takes for respondents to complete a survey can have a significant impact on the cost of online market research - - particularly in B2B market research as compared to consumer market research. Most online market research firms can provide pre-launch advice on survey length, and survey hosting vendors with survey timers can use this tool to help pinpoint the length. In addition, the programming structure of the online market research survey as well as the speed of page views and data inserts can impact survey length. Online market research firms should be able to offer helpful advice on programming options and offer suggestions on best use of skip logic, radio tables, and other features to reduce survey length.

Marc I. Tillman is a member of the professional services staff at Amplitude Research, Inc., a full-service online survey company headquartered in Boca Raton, Florida.

[Sep 21, 2009]