Who Are The People Who Don't Respond To Polls? [View all]

[brooklynite]

FiveThirtyEight

“Can we trust election polls?” is a question that has reached a fever pitch in political junkie circles dating back to the 2016 election. One popular theory about why election polls missed in 2016 and 2020 is that Trump-friendly voters refused to respond to surveys, making Trump’s support among the population appear lower than it actually was. People give many anecdotal reasons for why this happened, but the big takeaway from this theory is that election surveys are undercounting Republicans and Trump voters.

Of course, many proponents of this theory lack data when making this assertion. The FiveThirtyEight/Ipsos poll, conducted using Ipsos’s KnowledgePanel, can shed some light on whether this is happening in 2022. Back in April, we launched a panel study with a group of about 2,000 Americans whose demographic makeup was representative of the U.S. The goal of the project was to see how their fears and beliefs changed in the six months leading up to the midterm elections. However, we’ve found this project has additional, unintended value. Following that initial survey, there was a large drop-off in participation in the second wave and a smaller drop-off between the second and third wave, after which participation largely plateaued. In total, there was a 23 percent decline in the number of participants between the first wave and the sixth and most recent wave (the results of this wave are forthcoming).1 This allows us, in a limited way, to examine something called “nonresponse bias” — that is, who is not answering surveys — and how it impacts polling data.

Our data indicates that some respondents who lean toward the Republican Party are less likely to take part in follow-up surveys. But we didn’t find ourselves in a situation where all Republicans were not answering, and we were able to find a few clues as to who exactly these Republican non-respondents could be.

The charts below show the share of various types of respondents from our initial survey who then took part in subsequent waves.2 By looking at each individual wave, we can get a sense of response rates for the different demographic groups we’ve surveyed: