“The total collapse of public opinion polls shows that this country is in good health,” E. B. White wrote in The New Yorker. “A country that developed an airtight system of finding out in advance what was in people’s minds would be uninhabitable…People are unpredictable by nature, and although you can take the nation’s pulse, you can’t be sure the nation hasn’t just run up a flight of stairs, and although you can take a nation’s blood pressure, you can’t be sure that if you came back in twenty minutes you would get the same reading.”
White did not write these words after the 2016 or the 2020 Presidential elections. They appeared in November 1948, just following the polling debacle of the Harry Truman/Thomas Dewey election.
In that election, George Gallup, guru of modern polling, employed a sampling formula developed in the 1930’s to predict that Dewey would receive 50 percent of the vote, Truman 44 percent, and third party candidates 6 percent.
When the votes were counted, the result turned the poll on its head: Truman netted 50 percent, Dewey 45, and the remaining 5 percent went to the stragglers.
Gallup was wrong for two main reasons: his sampling methodology was flawed and he made a whopping miscalculation. Assuming that voters’ minds would not fluctuate near the end of the campaign, Gallup stopped polling two weeks before the election.
This fiasco became memorialized in the photo of a beaming Truman holding up the Chicago Daily Tribune which bore the incorrect banner headline: DEWEY DEFEATS TRUMAN. Comedian Fred Allen’s quip — “Truman is the first president to lose in a Gallup and win in a walk” — was one of the most quoted lines of the time.
Statistician Andrew Gelman recently said “Polling is an inexact science. You have to be careful.” The two eternal challenges for polling, according to Gelman, are attaining a representative sample and accurately predicting turnout.
These factors have figured prominently in political forecasting since 1824, the year the first poll was conducted.
At the time, Andrew Jackson, John Quincy Adams, and Henry Clay were engaged in a spirited contest for the Presidency. There was no incumbent involved, since President James Monroe was retiring.
The Harrisburg Pennsylvanian newspaper conducted a straw poll (as in, holding up a straw to determine which way the wind was blowing) among the citizens of Wilmington, Delaware. Jackson easily bested both Adams and Clay –surprisingly, considering the rudimentary methodology, the poll was fairly accurate when the general election was held. In the election, none of the candidates secured enough electoral votes, so the House of Representatives eventually tapped Adams to take the office.
The 1824 survey held one other surprise in addition to its relatively solid prediction — the primitive approach of the straw polling proved to be very long lasting. Until George Gallup developed a more scientific paradigm which emphasized a small, carefully selected sample of voters, large straw polls were the way to go.
Gallup generated an epistemological earthquake when he correctly predicted that Franklin Roosevelt would defeat Alf Landon in the 1936 Presidential election. This call was an outlier: other organizations, some of which had sample sizes in the millions, anticipated a Landon victory. Gallup changed the face of polling with one election. He only questioned 50,000 people, but, because they were so well selected to represent the country’s population as a whole, his prediction turned out to be on the money.
The 1948 Truman/Dewey election was a misstep for Gallup. But pollsters subsequently tweaked their internal procedures and went on to enjoy a reasonably successful run with a few speed bumps along the way.
That is until the 2016 Trump/Clinton trainwreck. And then the 2020 Trump/Biden clash, which statistics sage, Nate Cohn, said was a bigger miss than 2016. The recent Gavin Newsom recall election in California and the even more recent New Jersey governor’s race were both misses for the industry.
In a perceptive piece written last year, Courtney Kennedy of Pew Research Center, detailed the travails facing contemporary pollsters. She noted numerous potential pitfalls — the difficulty of “weighting” voter variables, how the standard margin of error is underestimated, the slipping quality of methodology caused by the proliferation of new survey firms, and uncertainty surrounding voter turnout are just some of the obstacles she discussed.
Kennedy wrote: “People have many notions about polling…that are frequently false. The real environment in which polls are conducted bears little resemblance to the idealized settings presented in textbooks.”
Many may share E. B. White’s disdain for polls, but, as Gallup pointed out, they are vital in one respect. They give ordinary citizens, who have no public platforms, opportunities to speak out. These are important voices to be heard, especially when they are reported accurately.