|I was recently asked by a
client for an opinion about the validity of Web surveys. My immediate
inclination was to denigrate the methodology, in the process trotting
out the commonly heard negatives, e.g., not random enough, questionable
sample composition, hard to verify the data, etc.
But online research is
now beyond the "prove it" stage. The marketing research
community, if it chooses to be fair, will concede the point that there
is now an irreversible decline in traditional data collection, e.g.
phone or mail surveys. The techniques are too costly, too laborious and
subject to limited response rates.
Meanwhile, Web surveys
are estimated to have achieved a penetration rate of 15 percent to 20
percent of the target survey pool. That makes a lot of sense in an
environment where there are now in excess of 190 million Internet users
in the U.S. and Canada alone.
The objections to online
research are understandable and, to some degree, rational. But, in every
case, there are equally valid counter arguments or mitigating factors.
It's not random enough
— Neither, for that matter, is the telephone, given the degree of
phone screening that currently prevails and the ubiquity of voice mail
as a barrier to entry into the home or office.
Response rate —
Typical response rates will run 1 percent to 2 percent for mail, 10
percent to 15 percent for phone. For online surveys, that number grows
to as high as 30 percent to 35 percent where respondent names are drawn
from weighted panels. The response rate is even better for B2B research
where reliable lists are more readily available, e.g. clients and
They tend to lie on
the Internet — But so can they on the telephone. Ever wondered how
many administrative assistants fill out the mail surveys for their boss?
Internet respondents will freely reveal private demographic information.
Ironically, the Internet delivers a perception of anonymity that is
greater than the phone or snail mail.
Cost — Committed
Web survey user companies maintain that they are not yet seeing
significant cost savings although the average 20-minute phone interview
can cost in the range of $30 to $50 per completed interview vs. $7 to
$10 for one completed online.
This is where the online survey wins out hands down. Reports and
summaries are developed real-time and can be funneled directly to
management in a simple, easy-to-digest fashion (resplendent in color,
with graphics and charts). Traditional phone/mail surveys require
labor-intensive data collection, tabbing, summary and distribution — a
decidedly non-real-time scenario.
Bad Lists —
Unquestionably, e-mail lists are subject to corruption unless the online
research is conducted using names drawn from a reliable panel or
verified database. The risk of bad research is real and palpable and,
although it attracts more visibility in the online universe, the disease
infests every environment that depends on a list of names for its
The one area where online
research has not proven popular is in the qualitative arena,
specifically online focus groups.
John Maben of the
Denver-based marketing research firm TwentyTen Research (and a
facilitator of online surveys) believes the jury is still out on the
subject. Notes Maben, "It's still a difficult process — lots of
typing and a chat-room environment that's not always conducive to
delivering rich diagnostic feedback."
Even though improved
instant-messaging software and Web-casting technology have enhanced the
process, there is still something about "seeing the whites of their
focus group interaction will always provide a degree of personal
involvement and insight that the online environment has difficulty
matching — even for the fast-fingered typist.
Joel Friedman, CEO of
SurveyWriter.com, says, "There can be no doubt as to the validity
and growing acceptance of the Web survey. Use of the methodology hasn't
grown faster because most market research firms are conservative in
their outlook and practice."
firm has conducted more than a million online surveys for more than 150
clients including the likes of Hewlett-Packard, Frito-Lay, All-State
Insurance and General Mills.
Alf Nucifora is an
Atlanta-based marketing consultant. Reach him at firstname.lastname@example.org.