538: Calm Down About Those Virginia Polls, Folks

[RandySF]

A Monmouth University poll released Tuesday generated a ton of headlines. It found Republican Ed Gillespie leading the Virginia gubernatorial race by 1 percentage point, 48 percent to 47 percent. Democrat Ralph Northam has led in most surveys of the race, and if Republicans win Virginia with President Trump so unpopular, you can expect a full-blown freakout among Democrats. Adding to the confusion: Quinnipiac University released a poll on Wednesday showing Northam up 14 percentage points, 53 percent to 39 percent.

So what the heck is going on in the Virginia governor’s race? Nothing. The split between the Monmouth and Quinnipiac results is big, but it’s not unnatural. In fact, it’s a sign that pollsters are doing their job.

Polling averages work best when pollsters are working independently. You have different pollsters using different methods and making different estimates of the electorate, and you get a more accurate picture of the race by averaging their results together than by looking at any individual poll. It’s kind of like the old “wisdom of the crowd” principle.

That doesn’t work if pollsters “herd” — which my colleague Nate Silver defined as “the tendency of polling firms to produce results that closely match one another.” When pollsters release results that are closer to each other than is statistically plausible, it may make individual polls more accurate, but it makes the average less so. That is, there should be a big spread among polls of the same race. Unfortunately, herding happens, particularly as Election Day approaches.


https://fivethirtyeight.com/features/calm-down-about-those-virginia-polls-folks/?ex_cid=story-facebook

[FSogol]

1. It will all come down to turnout. The GOP base who thinks that blocking women's healthcare and

protecting confederate statues are Virginia's most important issues are going to show up on election day. They always show up.

We need to get our side to show up in droves.

[Wounded Bear]

2. K & R ......for truth...nt

[world wide wally]

4. Yuuuuuge droves!

[ProfessorGAC]

3. Boy Do i Not Agree

The idea that different methodology producing results differing by this multiple of margin of error means either; one of the methods is badly flawed, or both methods are badly flawed.

I don't accept the "herd" hypothesis. If these were good methods with adequate sample size, two polls should not differ by this much.

[lagomorph777]

5. Pollsters are desperately trying to find good methods

The world of polling has been really hurt by the advent of cellphones, social networking, people zoned-out on their phones all day, drastic differences in turnout levels, unpredictable results of voter suppression and theft-by-voting-machine.

They should be experimenting; that will inevitably cause a large standard deviation across pollsters.

[ProfessorGAC]

6. I Know People Who Work At Pew

I know of nobody that would experiment and then publish the results. The experimental polling would be to refine the methodology that would then result in publishable results.

To do otherwise is statistically invalid and sloppy science.

And three of your reasons have nothing to do with polling. People polled can still have a preference and suppression might stop them from acting on that choice, but that would apply to any pollster.

They're only measuring preferences. People have preferences whether others have conspired to take away their voting rights, or not. Same with theft by voting machine. The people still had a person for whom they were going to vote. The machine stealing the vote doesn't change the preference identified in the polling. And all the instant communications stuff influences preference, but it doesn't affect the way the preference is measured.

[lagomorph777]

7. Those external problems do affect the utility of a poll for predicting outcomes.

And that does affect the credibility of the pollster. Of course they have to worry about them.

And, my belief is that, like it or not, all polls are experimental when the society is being broken down by corruption. You can't run experiments quickly enough when each election is tainted in new ways. And elections are the bottom line - can we reasonably predict them, can we reasonably test their validity?

[ProfessorGAC]

8. Well, We Disagree Greatly

First, I was never talking about outcomes and flaws WITHIN any poll. I was questioning any methodologies that cause differences this profound BETWEEN polls.
And, you obviously don't care how the experts do it & are simply willing to accept these differences as extrinsic influences pollsters are powerless to avoid
I think that's totally wrong

[lagomorph777]

9. Thanks for your presumption about what I care about.

Though you happen to be entirely incorrect about that too.

I'd love nothing more than to see polls we can trust. Some pollsters are clearly dishonest (e.g. Rasmussen), some are lazy, but I believe most are genuinely struggling to figure out how to deal with a collapsing political culture, combined with revolutions in technology and lifestyle. Those things add up to extreme difficulty in taking random samples, in getting people to answer honestly when they hold shamefully immoral views, and in getting people to believe poll results that never seem to match election results, nor do they match other poll results.

I'd love to be able to say: "See, the polls were definitive. That election was (or was not, in the most recent case) valid!" I'd love to see some published science on how to make polls better. I have a scientific mind, and scientific method matters a lot to me. Social science is so much squishier than the sciences I normally work with, so it is far more difficult for me to suggest a method.

Describe to me your perfect poll methodology, the one that will make pollsters slap their foreheads and say "Why did't we think of that?"

[ProfessorGAC]

10. Whatever!

Don't know much about statistics I see!
That's your problem!

[lagomorph777]

11. Sampling theory is not only about the math.

Though I am fairly familiar with the math too.

If I sample 100,000 people instead of the usual 1,000 they use, it won't improve the result unless I sample a representative 100,000 people. That's the problem here, not the math, Prof. Gack.

[Tobin S.]

12. It seems to me that accurate polling would depend on everyone or almost everyone telling the truth.

That alone, to me, would seem to make them unreliable. And what if there was actually an effort to some extent by a good portion of the sample population to actually try to throw the poll off by lying. A little tin foil hat there, but it seems like it could be a possibility to me.

I'm no expert on the issue. Just a few ideas I had.

[Awsi Dooger]

13. Shy Tory Factor

IMO, an American version of that is very real and ongoing and unlikely to cease anytime soon. But these rightwingers aren't shy as much as angry. They link polling organizations as a form of media and therefore not trustworthy.

I would expect polling in this cycle and upcoming cycles to demonstrate a red shift, when there is a discrepancy.

The double digit Northam leads make no sense to me. Virginia is trending our way but the ideological split was still 26% self-identified liberals to 33% self-identified conservatives last year. The nation was 26-35. So the state is slightly more blue than the nation itself.

Outside presidential years the electorate tends to be more male and more conservative. Instead of 53% females, like 2016, it probably will be 51% in the governors race.

I think Northam will win but inside the polling average.

And I have no idea how wild disparity from one firm to the next is touted as a positive toward industry methods. Maybe they were short one stupid article for this month.

[Yavin4]

14. How does a pollster compile a representative sample?

Representative sample of the entire population, of registered voters, of likely voters are all three very different sampling methods. Each one can describe the larger population while at the same time not be able to predict who will win the election.

[lagomorph777]

15. Exactly! Even defining "representative" is tricky

especially with distrust of media, possibly fear of totalitarian-style retribution against opponents, etc.