Commentary Magazine


Five Reasons There Are Bad Polls

This morning, CBS and the New York Times announced excitedly that their new swing-state poll (conducted by Quinnipiac University) in Ohio, Pennsylvania, and New Jersey showed a substantial margin for Barack Obama in all three. The problem: The poll’s results are preposterous. We know this not because it shows Obama leading but because its “internals” are hilariously out of whack in relation to vote totals in 2008 and 2010. For example: The poll has the president winning among independent voters in Pennsylvania by 22 points, 58-36. It is difficult to find state-by-state exit poll data from 2008, but in that triumphant year for Obama, he won independent voters nationwide by 6. 2008 exit polls had Obama winning by about the same margin in Pennsylvania—but in 2010, exit-poll data found Republicans winning the independent vote nationwide by 18 points. Could that have simply swung back all the way to 2008 numbers in Pennsylvania? All but impossible.

The poll itself reports that Democrats outnumber Republicans in Florida by nine points. In 2008, when Obama won the state by 2.5 points, the Democratic advantage was 4 points. Do we really think there are more Democrats in Florida in 2012 than there were in 2008? Even more telling, those polled say they voted for Obama by a margin of 13 points in Florida. Same for Ohio’s sample. Obama won Ohio by 4; those polled today say they went for him by a margin of 15 points. I could go on and on, and have on Twitter (@jpodhoretz, if you want to look). The data are so off-base that it might make you wonder why CBS and the Times didn’t trash the poll and start over.

So why didn’t they? Five theories (after the jump).

1. They like the result. This is what most conservatives will say and think. The poll is skewed because the liberal media want it skewed, and they do thinks to make it come out that way. Let’s give them credit for professionalism and skip this one, because it cannot be verified.

2. Polling is expensive. Media organizations aren’t rich the way they used to be, and they don’t have the luxury of ditching a bad poll that is simply the result of statistical noise and other difficulties.

3. They blew the sample for other reasons. Let’s say you want a poll to provide a decent measure of opinion among minorities, who skew overwhelmingly Democratic. A successful random sample will not give you sufficient numbers of minority voters to demonstrate anything. So you have to “weight” your sample. How you “weight” is where polling becomes less of a science and more of an art—and a pollster’s sense of the composition of the electorate will only be proven correct by the result in November. But in a relatively small sample of voters—and samples have been shrinking over the years as people are less patient with unsolicited phone calls—weighting minorities may have the result of skewing a poll’s results in the direction of Democrats. The only pollster in 2012 for whom that has not been consistently true is Scott Rasmussen.

4. They polled on the wrong days. There are well-known polling problems in summertime—since people travel a lot—and on weekends—when certain sectors of voters might be attending church or church functions rather than being around a phone. All of this tends to make it more likely pollsters will be talking to people who spend inordinate amounts of time at home, and thus provide a false portrait of the electorate.

5. Polling is dying. I alluded earlier to difficulties in getting people to talk on the phone. There’s also the rise in cellphone usage and in general, the greater fluidity of the American public, which is simply not as housebound as it was when Gallup figured out most of what we know about how polling should work in the 1940s, 50s, and 60s. All of this represents a logistical nightmare for the polling business, which must therefore resort either to far more expensive ways of finding the right people to sample (which media organizations are increasingly unwilling to pay for) or to more “weighting,” which just turns pollsters into people who guess things, and whose guesses are as susceptible to biases and opinions as anyone else’s.

None of this, by the way, should make Mitt Romney dismiss this poll’s results. If he were really in command in this race, he would be doing better despite the structural advantages the Q poll and others present to Democrats. Basically, there’s no evidence he’s making the sale.

Join the discussion…

Are you a subscriber? Log in to comment »

Not a subscriber? Join the discussion today, subscribe to Commentary »

Pin It on Pinterest

Share This

Share This

Share this post with your friends!

Welcome to Commentary Magazine.
We hope you enjoy your visit.
As a visitor to our site, you are allowed 8 free articles this month.
This is your first of 8 free articles.

If you are already a digital subscriber, log in here »

Print subscriber? For free access to the website and iPad, register here »

To subscribe, click here to see our subscription offers »

Please note this is an advertisement skip this ad
Clearly, you have a passion for ideas.
Subscribe today for unlimited digital access to the publication that shapes the minds of the people who shape our world.
Get for just
Welcome to Commentary Magazine.
We hope you enjoy your visit.
As a visitor, you are allowed 8 free articles.
This is your first article.
You have read of 8 free articles this month.
for full access to
Digital subscriber?
Print subscriber? Get free access »
Call to subscribe: 1-800-829-6270
You can also subscribe
on your computer at
Don't have a log in?
Enter you email address and password below. A confirmation email will be sent to the email address that you provide.