Sam Wang (Princeton): "Gallup's results are biased by party ID."

[swag]

http://synapse.princeton.edu/~sam/pollcalc.html

"Relevant to the current Gallup controversy: Here is a table of Gallup national presidential polls, along with Party ID statistics for each poll. The GOP-Democratic margin in the poll correlates quite closely with the Bush-Kerry margin. In fact, the correlation coefficient is 0.73 (r2=0.53, P<0.001).

Put into lay terms, this means that the Party ID gap can account for over half the variation in the reported margin. A linear fit between the two is near 1, which means that on average, every extra Republican in the sample added one to Bush's margin. This analysis provides strong support to the idea that Gallup's results are biased by party ID. You should forward this link to your favorite reporter."

[crispini]

1. I heard Gallup's rebuttal on NPR this morning.

They said that they do their random calls, and FIRST used the questions that determine likely voters, and THEN they ask the voter who they're going to vote for, and THEN they ask the party. They pointed out that people planning on voting for GWB are more likely to identify themselves as Republican.

Leaving aside the question of how "likely" voters are determined, the rest of this actually does seem kind of logical to me.

Anyone got any more details on their methodology? Is this in fact the case?

[cheshire]

2. Funny I read both of these and all I got was BLAH BLAH BLAH or BS

[Mike L]

5. So Gallup is saying that people they poll lie about which party they are

registered with? I can see them fudging about which party they most closely associate with, but registration is something else.

I do think that the Russian school hostage killings had an effect on people's opinion. I remember reading the Republican convention news on the same front page as the Russian school story and thinking, 'Damn, could the timing of this tragedy be any worse?'

[crispini]

12. No, Gallup's not saying that people LIE (I don't think)

Not all states require you to register. And, it's kind of human nature to not necessarily know your registration. I'm talking now about Ordinary Variety of Human, not Extremely Political Variety of Human inhabiting DU and the like.

[progressivebydesign]

6. It means they are calling an inbalanced number of republicans..

.. they need to look at their calling methods, if they are getting mostly republicans.. because most people who vote are NOT republicans. Perhaps Republicans are most likely to be sitting home by the phone.. the dullards.

[Lex]

8. Maybe I'm thick-headed but don't statisticians have

to normalize their data?

In Gallup's case, they simply don't care if they've called far more Republicans than Democrats. They say their callers are "random" but if in fact, they are getting more Repubs than Democrats on the phone, then their supposedly "random" data isn't normalized properly and doesn't reflect accurate information we already know about the voting population.

Arghhh. Maybe this is why Statistics in college always drove me nuts.

[On the Road]

10. That Addresses One Issue, But Not the Main One

at least in this campaign. It may be perfectly true that Republicans are more likely to give party ID, but Republicans are still overrepresented in the three-way split (R/D/Ind). Republicans had 39-40% of the sample, as opposed to about 36% of the voting population in 2000. Democrats calling themselves independents would lower the Democratic percentage, but not raise the Republican %.

That suggests that Republicans are overrepresented in the population contacted by the pollsters.

[On the Road]

3. Wow -- It Looks Like Kerry by 4%

If you equalize for party ID (zero on the baseline) and then look straight up, it crosses the regression line at about -4%, which is the Bush lead.

That means a 4% Kerry margin, no?

[underpants]

4. Sssssssssssssh ribo

they might near the landslide coming

[ItsMyParty]

7. I was called by Gallup within this last year and it was way at the

end that they asked party affliation, etc. BUT, here's what I truly don't get. If they "happen" to call up 1,000 households and "happen" to get 850 registered republicans then how can a poll work at all?? Maybe not all 850 are going to vote Bush; but probably 849 are!! One would think they needed to call and call, write down the stats and at the end when they have roughly 300 repubs, 300 dems and 300 indpendent replies (and it might take 10,000 calls to get that number), then reach into the repub/Dem/Indep barrel and pull out 300 responses each. I guess what I'm trying to say is how can you base a poll on which party happened to answer the phone more that day????---that has to do with phones not voting trends.

[Lex]

9. See--that's what I'm thinking too!

Even if you try to get 1000 "random" people on the phone, if your collected data shows that you in fact ended up with 85 percent Republicans, then you can't just say Bush is ahead by so many points.

You have to normalize the data based on what we already know of voting population and patterns.

Yikes, I'm trying to remember this stuff from Stats class in college which was 15 years ago.

[DemocratSinceBirth]

11. Gallup Is Basically Saying Party ID Is Fluid

while the majority of political science research suggests it is constant...

Party ID is not as constant as race, income, education, et cetera but it's more constant than Gallup suggests...


It's an interesting academic debate....