How Do Really You Know?
When you write a weekly column like this,
you're bound to get an assortment of reactive mail. Sometimes it's flattering.
Sometimes it's threatening. And sometimes, it's even on topic.
Last week, a Mr. Richard Feder from Fort
Lee, New Jersey, zapped in this question: "I'm wondering if you could
tackle the topic of doing demographic surveys on your site. How does one
do it right? What are the pitfalls to avoid? How do you know if the answers
Well, Mr. Feder, I certainly can tackle
that topic. In fact, I'd be willing to wrestle it to the ground and pummel
it for you. And what better way to begin the pummeling than by pounding
out Rule #1, the Big Kahuna of all research rules:
Thou shalt keep it neutral at all times.
You see, the biggest goof do-it-yourselfers make is tipping their hand
when soliciting data from respondents. The way you ask your questions
is even more important than the veracity with which your respondents answer
them, because people with fake names have nothing to lose. On the other
hand, anyone with any name will be led astray if you ask the question
in a leading or influential manner. After all, which of these questions
do you suppose would be answered more truthfully:
1. My neck is (A) Small (B) Medium (C)
Large (D) Extra Large
2. The collar size I wear most often is
(A) >15 (B) 15 (C) 16 (D) 17+
Okay, so neck sizes aren't terribly stimulating.
But if you're in the rag business, this is can't-put-it-down kind of stuff.
One look at question #1 ought to tell you that it's way too personal,
because if focuses on the respondent's self-image. Chances are -- and
I know I'll get mail on this -- male respondents will edge toward the
Extra Large option, whereas the fairer sex will tend to gravitate toward
the more diminutive. When you shift the same information over to the SHIRT,
however, you're shifting the respondent's attention away from him or herself,
and on to the garment.
Sneaky, eh? But it sure does net you the
information you're after, in a much more reliable way.
Okay, so now that your survey is squeaky
clean from bias, how do you know your answers are truthful? Easy! MORE
TRICKERY! Oh, come on. You do this every day to find out if that guy or
girl you ran into at the library has a significant other. You can be just
as clever when it comes to verifying the demographics of your respondent
audience. It mostly depends on how you bait your victim.
1. MAKE IT SHORT: Keep it really simple
and quick. After waiting for your 154K graphic to load, the last thing
anyone wants to do is spend more time at your site doing things they don't
want to do. Four, five questions tops.
2. DON'T ASK FOR REDUNDANT INFORMATION:
Don't waste your questions on things like Name and E-Mail Address. People
regard that as very personal information. If you have a Submit form, it
can trap the user's name and e-mail anyway (provided their browser's configured
that way). If they don't fall for that, don't worry, because of #3.
3. KEEP YOUR EYE ON THE BALL: If you want
e-mail addresses, ask for those. But if you want information, don't
worry if you don't get e-mail addresses. Many people will give you better
information if they can stay anonymous.
4. BRIBES WORK BEST FOR E-MAIL ADDRESSES:
Hey, if you're offering a free something or other, people are going to
tell you where to send it. To verify, make sure your delivery process
is a two-step process, requiring them to send you a confirmation of their
5. MAKE THEM ONE OF THE CROWD: Nobody
likes to feel they're the only guinea pig in the lab. By placing a Guestbook
on the site -- one with a public comment section -- you can lull your
respondent out of their garden-variety internet suspicions by allowing
them to view previous respondents' comments. The more friendly your site
appears, the more likely your prospects are to let their guards down.
6. MAKE IT FUN: Come on. It's the least
you can do for them helping you out. If you come off too serious, they'll
think you're fronting for the CIA.
7. SUBMIT/E-MAIL THE QUESTIONS: Once you've
got them answering the questions, make it simple to get them out of their
faces. Click a Submit button and send it on its way to some safe harbor
where you can analyze the heck out it.
So Mr. Feder, if you're still wondering
whether anyone can get 100% truthful answers from a survey, the 100% truthful
answer is no. But don't feel bad -- even CNN, CBS, Newsweek and TIME magazine
polls carry a margin of +/- 3%.
And least that's what they admit to. Who
knows if they're telling the truth?