A (sad) tale of two polls. Actually, of one poll.

[NYC_SKP]

27. Phone polls are cheap to conduct but notoriously unreliable.

Landlines versus cellphones, caller ID, working and busy people versus people doing nothing sitting at home, or getting far more homebound retired folks, any number of problems can really screw with the outcome of a phone poll.

Then there's the matter of how the questions are asked.

And what qualifies a person as a "liberal" versus a registered Democrat? Is that a self-identification thing?

Do people ever lie during phone interviews? I know I have.

Abraham J. Wyner, Wharton professor of statistics, notes that “while the value of a good answer is worth the cost to get it,” it’s becoming more expensive to do a proper poll at a time when “the population is indifferent.” For instance, telephone polls are considered among the most reliable when it comes to following statistical models and obtaining a scientific random sampling. “But in the era of Caller ID, many people choose not to answer their phones if they see an out-of-area number on their machines.”

Indeed, a study conducted by the Pew Research Center in 2004 found that “more Americans are refusing to participate in telephone polls than was the case six years ago,” due to a growing number of unsolicited telephone calls and because potential respondents “are armed with increasingly sophisticated technology for screening their calls.” A typical survey that employed standard techniques used by opinion polling organizations obtained “interviews with people in fewer than three-in-ten sampled households,” representing a decrease of about nine percentage points from the late 1990s, the study reported.

The American Association of Public Opinion Research, an organization of public opinion and survey research professionals, points out that the survey research typically conducted by political and marketing polls is not covered by the recent “Do Not Call” registry, which was established by the Federal Trade Commission in June 2003 to meet the requirement of the Do Not Call Implementation Act. The law made it illegal “for telemarketers to call consumers with whom they did not have a prior business relationship. The FTC exempted survey and opinion research because it is a critical part of making and monitoring policy decisions.”

While pollsters have always grappled with the person who refuses to answer questions after they pick up the phone, today’s pollsters are dealing with what Johnston calls “the silent refusal” — the person who just decides not to pick up the phone. And it’s a big problem. “Willingness to answer the phone, to a great extent, is quite independent of other characteristics, which includes being interested in politics,” says Johnston. “The kinds of people who answer political surveys now, compared to 40 years ago, are the more interested stratum. You got more apathetics, more marginally interested people 40 years ago than you do now.”

more at: http://knowledge.wharton.upenn.edu/article/polling-the-polling-experts-how-accurate-and-useful-are-polls-these-days/