New to Online Surveys? Try It, You’ll Like It
by Catherine Giordano, President, AnswerSearch, Inc.
An overview of the fastest growing market research methodology--online surveys: Compares online research to traditional methodologies and provides important tips for maximizing the success of online survey projects.
Online research is a proven, trusted, and widely used methodology.
The use of online research to replace traditional research methods has been growing. According to Inside Research (January 2006), almost one third of all spending for surveys is for online surveys.
Inside Research reported that spending for online research has grown every year since its inception.
Online research has been proven to be valid.
Research shows the online data collection almost always leads to the same business decisions as traditional data collection methods.
Online research is suitable for most, but not all, research projects.
The main criteria for determining if a survey is appropriate for the internet are:
- The survey is suitable for a self-administered questionnaire.
- The target audience is reachable via email.
- The population with access to the Internet reflects the client’s audience.
- Open-end questions are minimal, since there is no interview present to probe for in-depth answers.
Online research has many benefits over traditional methodologies.
The obvious advantages of online research are quick turnaround and low cost. However there are many other advantages.
Online research is respondent-driven. Respondents complete the survey at their leisure on their own schedule. They can choose a time when they are not distracted. Additionally, online research does not engender the negativity that can arise from the intrusiveness of phone and mall solicitations for cooperation.
Respondents may be more cooperative and more honest because they have greater anonymity for questions dealing with sensitive topics.
Visuals can be embedded in the questionnaire or a link can be inserted at the appropriate points, putting online research on par with personal interviews with respect to the ability to use audio-visual stimuli.
The response rate for online research studies is often superior to that currently obtained with traditional methodologies.
The research industry has suffered from declining response rates for years. CMOR (The Council for Marketing and Opinion Research) reported 12% response rates in 2002, down from 16% in 2000 for telephone and mall studies. In contrast, online research sometimes reaches a 70% response rate.
Incentives can improve response rates. A cash incentive is the most effective motivator. AnswerSearch, Inc. has had success with using “sweepstakes” and with using electronic $10 Amazon.com gift cards offered to each respondent who completes a questionnaire.
Additionally, response rates can be improved by the wording of the invitation, particularly the subject line. The subject line should entice to recipient to read further. Additionally, a sponsor that is familiar to the respondents improves response rates. And, of course, care should be taken to avoid using words or phrases that are likely to set off spam blockers.
A brief questionnaire with simple straightforward questions will improve response rates.
The non-response bias with internet research is no different than that found with traditional methodologies.
Certain population groups have historically been less receptive to cooperate with surveys. Research conducted by e-Rewards (a company providing email panels) shows that the demographic patterns of non-response are the same for internet research as they are for traditional research methodologies.
The groups with below average response rates are older people, less educated people, lower income people, and sometimes males.
However, non-response bias is not always present. If invitations are sent to a representative sample, the probability of a balanced response is high. AnswerSearch, Inc., a full service market research company, recently conducted a mail study done with a database sample and obtained a 13% response rate. Because it was a database sample, demographic and behavioral information was available for the entire outgoing sample. There were either no differences or only very small differences between responders and non-responders on 12 demographic and behavioral characteristics.
If non-response bias is suspected, there are ways to minimize this bias. The outgoing sample and the incoming responses can be managed to achieve representativeness by setting requirements for eligibility and quotas for demographic and behavioral characteristics. Data can also be weighted to match known parameters.
Online interviewing allows for the use of various types of samples.
- People who have been pre-recruited to participate in research projects
- Panelists provide demographic and product usage, etc.
- High response rates; usually from 15% to 70%
- People, usually customers, who have become part of a company database
- A variety of information about the database members is available
- Response rates are usually in the 5% to 15% range
- People who have agreed to receive email messages related to a topic of interest, but not necessarily to participate in research
- Respondents are “pushed” to the survey site by responding to an email invitation
- No information on respondents available
- Usually low response rates—1% to 2%
- People are “pulled” to the research site by banners and ads
- Little is known about them except the information they may provide when they respond
- Uses a mix of the above four types of samples
A final word of caution
The apparent ease and simplicity of online surveys may encourage a DIY (Do It Yourself) mindset. This would be a mistake. The same expertise that goes into study design, questionnaire design, and analysis for traditional methodologies is required for online surveys as well. It is still important to use an experienced market research professional for online research in order to insure valid actionable results.