Another day, another set of polls. One says John McCain is now tied with Mitt Romney for the lead in New Hampshire. But that does not comport with the findings of other polls, which have Romney ahead by seven or ten or twelve points. Barack Obama has closed the gap with Hillary Clinton nationally and trails her by seven — or is twenty points back. And then there is Iowa, where Hillary is ahead, Obama is ahead, or Edwards is ahead. Huckabee might be ahead by fifteen or by two.
Results this varied are what has caused most people to begin to rely on an average of all polls being taken. The Poll of Polls most frequently cited is the one at realclearpolitics.com, which popularized it. Mark Blumenthal of pollster.com explains why a poll average works with an analogy to darts. Let’s say you’re playing darts and you don’t know where the bullseye is. You can figure it out by looking at the pattern of the holes created by other dart-throwers, which in an effort to reach the bullseye will actually create a picture of it in absentia.
The problem with that analogy is that there isn’t just one bullseye in a primary poll. There are five or six. Each candidate a pollster asks about is a bullseye. And with all these other possible bullseyes, the pattern of the holes around each one of them is not going to be anywhere near as distinct. It stands to reason that if you ask 500 people about a choice between A or B, you’re going to get a large number for A and a large number for B, and that one of the two will be larger than the other. If you ask 500 people about a choice between A, B, C, D, E, F or G, you’re not going to get big numbers for any one of them but relatively small numbers for all of them. And the pattern created by each choice — corresponding to a single dart — is more like an impression than a solid pattern.
Add to this uncertainty the fact that 14 percent of Americans now only use cellphones. Pollsters haven’t figured out how to factor in cellphones, so that’s 14 percent of the potential electorate missing from their sample to begin with. Add further the fact that many people — no one has a number, but it is significant — now hang up on people they don’t know or don’t answer the phone when their Caller ID offers an unknown phone number, and you have another segment of the population that is offline.
Now consider Iowa and New Hampshire. These are states whose residents are being bombarded daily by phone calls from campaign volunteers, campaign staffers, and recorded messages from candidates. As Richelieu, a campaign guru who posts on the Weekly Standard’s Campaign blog, puts it:
Polling right now in Iowa and New Hampshire is a technical nightmare. Every three minutes the average voter’s phone rings with somebody coaxing them to trudge out into the snow and attend an Edward’s meeting, go to a coffee with one of Romney’s sons, or sign up for a Huckabee prayer circle. Not to mention the endless pre-recorded “robo-call” phone messages from various crank interest groups grinding their axe on some issue. With your phone ringing two dozen times a day with a political call, it is not easy for the 35 different media and private pollsters each trying to get a sample done each night. Voters don’t answer the phone or refuse to play along when they do answer. Which means response rates go way down, samples tilt away from a statistically reliable random frame of the population, and results go bad.
And now for the most important part: Turnout in the Iowa caucuses is expected to be somewhere around…this is serious…five percent. That means five percent of the state’s universe of Republicans will attend a Republican caucus meeting, and five percent of the state’s Democrats will attend a Democratic caucus meeting. According to Blumenthal of pollster.com, “The historical high for turnout in the Iowa Caucuses was 5.5% of adults for the Democrats in 2004 and 5.3% of adults for the Republicans in 1988.”
Now here’s what this means. For a poll to achieve a measurable degree of scientific accuracy, a pollster “would need to screen out nine out of ten otherwise willing adults in order to interview a combined population of Democratic and Republican caucusgoers strictly comparable in size to past caucus turnouts.” Because no pollster can afford to do such a thing — to reach thousands of people and then discard the results from 90 percent of the phone calls — each polling firm has to come up with its own theory of how best to locate and identify likely voters in sufficient numbers. That’s why, Blumenthal says, the results of each poll vary so wildly.
So. People who are polled are offered six options. People use cell phones exclusively. They know they’re getting political calls and don’t answer the phone if they’re at home. And only five percent of voters in each party actually turn out in Iowa.
So. Still confident there’s a Huckabee “surge”? Or that Romney has ended the Huckabee surge? Or that Obama is gaining on Hillary? Or that Edwards can’t win? If you are, I would like to sell you this.