Welcome to DU! The truly grassroots left-of-center political community where regular people, not algorithms, drive the discussions and set the standards. Join the community: Create a free account Support DU (and get rid of ads!): Become a Star Member Latest Breaking News Editorials & Other Articles General Discussion The DU Lounge All Forums Issue Forums Culture Forums Alliance Forums Region Forums Support Forums Help & Search

OKIsItJustMe

(22,413 posts)
12. A wild supposition
Fri Nov 1, 2024, 08:09 PM
Nov 2024

Suppose, there was some systematic sampling error. (Say… a certain demographic does not like to reply to polls, or is ashamed to tell polsters the truth.)

https://www.pewresearch.org/short-reads/2016/11/09/why-2016-election-polls-missed-their-mark/

NOVEMBER 9, 2016
Why 2016 election polls missed their mark
BY ANDREW MERCER, CLAUDIA DEANE AND KYLEY MCGEENEY

The results of Tuesday’s presidential election came as a surprise to nearly everyone who had been following the national and state election polling, which consistently projected Hillary Clinton as defeating Donald Trump. Relying largely on opinion polls, election forecasters put Clinton’s chance of winning at anywhere from 70% to as high as 99%, and pegged her as the heavy favorite to win a number of states such as Pennsylvania and Wisconsin that in the end were taken by Trump.

How could the polls have been so wrong about the state of the election?

There is a great deal of speculation but no clear answers as to the cause of the disconnect, but there is one point of agreement: Across the board, polls underestimated Trump’s level of support. With few exceptions, the final round of public polling showed Clinton with a lead of 1 to 7 percentage points in the national popular vote. State-level polling was more variable, but there were few instances where polls overstated Trump’s support.

The fact that so many forecasts were off-target was particularly notable given the increasingly wide variety of methodologies being tested and reported via the mainstream media and other channels. The traditional telephone polls of recent decades are now joined by increasing numbers of high profile, online probability and nonprobability sample surveys, as well as prediction markets, all of which showed similar errors.








https://www.pewresearch.org/methods/2021/03/02/what-2020s-election-poll-errors-tell-us-about-the-accuracy-of-issue-polling/
MARCH 2, 2021
What 2020’s Election Poll Errors Tell Us About the Accuracy of Issue Polling
BY SCOTT KEETER, NICK HATLEY, ARNOLD LAU AND COURTNEY KENNEDY

Most preelection polls in 2020 overstated Joe Biden’s lead over Donald Trump in the national vote for president, and in some states incorrectly indicated that Biden would likely win or that the race would be close when it was not. These problems led some commentators to argue that “polling is irrevocably broken,” that pollsters should be ignored, or that “the polling industry is a wreck, and should be blown up.”

The true picture of preelection polling’s performance is more nuanced than depicted by some of the early broad-brush postmortems, but it is clear that Trump’s strength was not fully accounted for in many, if not most, polls. Election polling, however, is just one application of public opinion polling, though obviously a prominent one. Pollsters often point to successes in forecasting elections as a reason to trust polling as a whole. But what is the relevance of election polling’s problems in 2020 for the rest of what public opinion polling attempts to do? Given the errors in 2016 and 2020, how much should we trust polls that attempt to measure opinions on issues?1

A new Pew Research Center analysis of survey questions from nearly a year’s worth of its public opinion polling finds that errors of the magnitude seen in some of the 2020 election polls would alter measures of opinion on issues by an average of less than 1 percentage point. Using the national tally of votes for president as an anchor for what surveys of voters should look like, analysis across 48 issue questions on topics ranging from energy policy to social welfare to trust in the federal government found that the error associated with underrepresenting Trump voters and other Republicans by magnitudes seen in some 2020 election polling varied from less than 0.5 to 3 percentage points, with most estimates changing hardly at all. Errors of this magnitude would not alter any substantive interpretations of where the American public stands on important issues. This does not mean that pollsters should quit striving to have their surveys accurately represent Republican, Democratic and other viewpoints, but it does mean that errors in election polls don’t necessarily lead to comparable errors in polling about issues.

How is it possible that underestimating GOP electoral support could have such a small impact on questions about issues?

Why did we choose to test a 12-point Biden lead as the alternative to an accurate poll?

We created a version of our surveys with an overstatement of Biden’s advantage in the election (a “tilted version”) to compare with a “balanced version” that had the correct Biden advantage of 4.4 percentage points. The 12 percentage point Biden lead used in the “tilted” version of the simulation is arbitrary, but it was chosen because it was the largest lead seen in a national poll released by a major news organization in the two weeks prior to Election Day, as documented by FiveThirtyEight. Several polls had Biden leads that were nearly as large during this time period. The simulation, including the manipulation of party affiliation among nonvoters, is described in greater detail below.

Recommendations

0 members have recommended this reply (displayed in chronological order):

2-3% lead with a 4% MOE isn't exactly"getting crushed" nt Fiendish Thingy Nov 2024 #1
Great minds and all that... TwilightZone Nov 2024 #3
It isn't of course OKIsItJustMe Nov 2024 #5
It's the title of the video - Tim Miller named it Earthrise Nov 2024 #11
A 2-3% lead in polls that have an MOE isn't even remotely "crushed" TwilightZone Nov 2024 #2
Just check the poster's bio NoRethugFriends Nov 2024 #4
Sources OKIsItJustMe Nov 2024 #6
There is too much value being put on MOE Deminpenn Nov 2024 #7
MOE isn't a theory. TwilightZone Nov 2024 #8
It's your theory and conclusion Deminpenn Nov 2024 #9
No clue where you got that idea. TwilightZone Nov 2024 #10
I understand MOE is a statistical calculation Deminpenn Nov 2024 #13
A wild supposition OKIsItJustMe Nov 2024 #12
Latest Discussions»General Discussion»MASSIVE New Poll Shows Tr...»Reply #12