Are Pollsters Oversampling Republicans?

[ccharles000]

I have received a myriad of requests asking me to respond to this Seth Coulter Wells article at the Huffington Post on changes in the party ID composition of recent polls. Although the article itself is a good and thoughtful piece, some of the interpretations of it are not. So let's get a few things straight:

1. Polls that show changes in party ID composition are not "cooked", "rigged", or "biased". Changes in party ID composition may occur for a variety of reasons, including random chance, response bias, temporary changes in party affiliation, and longer-term changes in party affiliation. It is possible that, if the partisan ID composition of a poll shifts rapidly from week to week (more so than is likely from random chance alone, and more so than is precipitated from external events), this may be an indictment of a pollster's methodology -- that the pollster is having difficulty getting a good, random sample. But the notion that credible pollsters like Gallup or SurveyUSA are deliberately rigging their samples is patently ridiculous. Polling is a very competitive industry, with relatively low barriers to entry; they would go out of business in a hurry if they did this.

2. The decision a pollster faces is whether or not to weight its sample by party ID. In fact, the whole point is that pollsters like Gallup and SurveyUSA do not weight their samples by party ID -- they just tally the results, and let the chips fall where they may. So in some sense accusing them of "cooking" their samples has it backward; what you're really arguing is that they should weight their samples, presumably in a way that is more favorable to your preferred candidate.


http://www.fivethirtyeight.com/2008/09/lets-get-few-things-straight-party-id.html

[IndyOp]

1. I love 538.com! Thanks for posting this. You might want to trim it a bit -

- you are only supposed to post about 4 paragraphs from any online article.

:hi:

[ccharles000]

2. Thanks for telling me

[AtomicKitten]

3. Yes - simply by virtue of cell phones vs. land lines.

[IndyOp]

5. Are you suggesting that Democrats are more likely to have cell phones

and therefore less likely to be represented in the polls?

Maybe, but I am not sure about that - at all. Cell phone ownership is pretty ubiquitous these days.

:shrug:

[AtomicKitten]

8. Between the ages of 18-32, Obama's money demographic, yes.

http://pewresearch.org/pubs/901/cell-phones-polling-election-2008

Cell Phones and Voter Preferences

In the current poll, cell-only respondents are significantly more likely than either the landline respondents or the cell-mostly respondents to support Barack Obama and Democratic candidates for Congress this fall.

[AZ Criminal JD]

7. They poll cell phones.

Why do people still think they don't? Crazy.

[AtomicKitten]

9. Cell phone polling is problematic. Details inside.

On edit: I never said they don't do cell phone polling; I merely suggested (and as this piece states) it is problematic.

http://pewresearch.org/pubs/901/cell-phones-polling-election-2008

The more serious challenge to survey research posed by cell phones is the declining absolute numbers of certain types of respondents, most notably the young. In recent Pew Research Center surveys, only about 10% of respondents in landline samples are under age 30, which is roughly half of what it should be according to the U.S. Census. Young voters reached on landlines share many of the characteristics of the cell-only group, especially in terms of political views. That is why statistical weighting of the landline samples helps to correct for the absence of the cell-only. But the shortfall of young respondents in absolute numbers means that pollsters are limited in their ability to analyze differences within this age group.

[Bread and Circus]

4. I agree with indicting the Republican Party as a whole...George Bush is just a face man

for a whole system of bad ideas and corrupt practices. He's a sign of the disease, but he's not the disease itself.

One of Obama's framing problems is that he has mainly focused on George Bush, understandable because GWB has such bad ratings, he's an easy target. The problem is that GWB isn't running for President.

I actually think it's too late but Obama should have spent more time framing an argument against the whole Right Wing's collection of failures.

[grantcart]

6. phrigndumass has shown that they are not only oversampling Republicans

but that they are also underreporting AAs and younger voters.

[brooklynite]

10. Here's a simple answer...

The polls that were inaccurate last week are inaccurate this week. The pollsters increased their Republican ID share because they assumed (probably correctly) that Republicans were now more enthused and would be more likely to vote. This assumption doesn't mean their original Republican baseline was correct in the first place.