"Are Phone Polls Accurate?" (not really -- interesting study)

[villager]

A Quick Question

Are phone polls accurate?


The quick answer: They are accurate for those that respond, but the crucial question is how many actually responded.

The longer answer: Today, telephone polls have become a standard part of news reporting. Every scandal, major news event, or change in direction is heralded by an “instant analyses” of who is up, who is down and “What America thinks”. Often, the findings from these polls take on a life of their own, framing what people think about an event and shaping their response to it. Yet by social scientific standards these findings are often of poor quality, or worse, are misleading, representing what an outspoken minority feels rather than mainstream America.

Why misinformation is popular

Information from these polls is popular because it is relatively quick and inexpensive way to put the survey sponsors in the position of being knowledge brokers. In effect they have “created news.” It is in their best interests to present the information in a way that shapes headlines. The results are typically reported in a pseudo-scientific fashion, stating that the poll was based on 1000+ adult respondents in a national sample with a margin of error of plus or minus 3%.

What they don't tell you

The American public has been trained to assume that a national poll of thousands with a small margin of error accurately represents the opinions of the whole society, but is that true? In fact, the sample size and margin of error give no indication as to the more important information about whether the results accurately reflect the population it has surveyed. The margin of error simply reflects how well the responses cluster around an average score.

<snip>

It makes you wonder, whose opinions do these polls really reflect? While not all telephone polls are equally unreliable one rule of thumb is that those who do not report a response rate should be treated with a high degree of skepticism. Yet until the public stops believing in them, telephone polls will be tempting to those who, for a few thousand dollars, wish to see themselves as knowledge brokers.

http://hirr.hartsem.edu/research/quick_question27.html

[Feanorcurufinwe]

1. Nobody can poll me on the phone

because the only people I speak to on the phone are family, personal friends, or business clients.

[depakid]

2. Nice discussion of response rates (non-response bias)

It's not even arguable anymore that the corporate media flaunts misleading or downright unscientific poll results to create news and shape public opinion. Unfortunately, it works- which is why they do it.

[villager]

3. right -- "self-fulfilling polls"

...brought to you by the media division of Big Bidness...

[jdj]

9. Their only purpose is to manufacture consent.

They play on the instinct to conform by telling people in which direction to conform.

[villager]

15. almost used "manufacture consent" in the header...

...when I posted this. Yer right.

[peacefreak]

4. I was polled a couple of weeks ago

and it was obivous by the questions who sponsered it. Will you vote for *? Are you Pro-Life? Do you suport the right to bear arms. I felt like washing after. It was blatant.

[depakid]

6. Ya got push polled

of course, really effective push polling is much more subtle than that...

[wurzel]

5. Republican pollsters admit they poll 9 to 5.

When most Democrats are at work. Polls are a sham.

[villager]

8. be interesting to compare...

the parameters of how the various polls operate: i.e. when Zogby calls vs. when Gallup calls, the extent of their sampling (including how many refuse to answer, etc.)

Might give us a better sense of which polls to trust. If, indeed, we can trust any of them.

[59millionmorons]

7. It's all we have

http://reuters.com/newsPhotoPresentation.jhtml?type=politicsNews&imageID=1001304544

[mdguss]

10. The 4th Estate Won't Tell You:

But you will learn in Political Science 101 that we are reaching a saturation point with polls. So many people get solicited for so many things, that many (perhaps most) average people won't go along with them.

The result is kind of curious: only people that believe passionately in one side or the other actually vote in polls. So the polls end up being near-dead-even, which allows the 4th Estate to recycle canned stories on a "divided electorate." Another problem is turnout models a variance in 1 percent in turnout can swing a poll 4 or 5 percent. My guess is CNN has a turnout assumption lower than the Zogby poll. The lower turnout assumption would mute any new voters who were mobilized by the convention.

Zogby nailed New Hampshire and the general election in 2000. He was pretty much right about the 2002 elections as well. I'd put more stock in his poll than the others.

Remember, "Members of the 4th Estate, your reign is about to end," John Kerry.

[villager]

12. good points...

...the whole "likely voter" thing is another statistical flaw in a lot of these polls, I think... especially in this election cycle...

[Tight_rope]

11. I don't trust them...

[central scrutinizer]

13. I teach statistics classes

and these are the questions we always discuss in class. I spend several days on how to lie with statistics, charts and graphs. I have an ongoing assignment every term where I have the students find an article or chart from a magazine or newspaper each week and critique it. USA Today is a rich source - their pictographs are often distorted. Polls are especially suspect: I tell them any poll that doesn't give the EXACT wording of the question (even then you can't tell if there was bias) the non-response rate and describe the methodology is worthless. Worse than worthless - it is fraudulent.

[villager]

14. I wonder if we can get a poll-critiquing meme...

...out there, for people to consider whenever these "statistics" are released. After all, wasn't poll-tweaking part of the GOP's deliberate misinformation campaign designed under the Gingrich "revolution?"