Data Science & Psychology Data Science applied to Values, Morals, Politics, & things that matter.

One of the many great points I took from Nate Silver's recent book, The Signal and the Noise, is that people are generally bad at dealing with uncertainty. We want the weather forecaster to tell us if it's going to rain tomorrow, not that there is a 30% chance, even though that's the right answer. Nevertheless, weather forecasters have failed enough, in part because they get feedback every day, that they insist on giving us probabilities. Perhaps pollsters haven't failed enough, as they still continue to insist that they can categorically predict who is going to vote and who isn't. Since voting records are largely public (whether you voted, not whom you voted for), a recent study by Todd Rogers and Masa Aida was able to actually look at how effective it is to ask a respondent whether they plan on voting. Below is a chart showing what % of people who reported each intention actually voted.

Percentage Who Actually Vote x Reported Intention in 2008 (from Rogers & Aida)

Pollsters can predict who votes and who doesn't above chance, but in the same way that poker players probabilistically predict the likelihood of winning a hand, accepting the inherent uncertainty, pollster predictions on who votes are also likely to be wrong much of the time.  Pollsters should realistically adopt the same tactics as poker players who multiply the expected outcome (the size of the pot vs. what they have to put in the pot) by the likelihood of the outcome (the odds of the right cards coming to win the hand) in order to determine their decisions.  Similarly, polling would be vastly improved if pollsters weighted votes by voting intention rather than categorically deciding that a person is or is not a likely voter.  What would be the result?

Right now (October 26, 2012) Romney is leading Obama in Gallup's survey 51-46 among 2700 likely voters, but they are tied 48-48 among 3,050 registered voters.  If one does the math, Obama leads Romney approximately 63-25 among unlikely voters.  If the pattern from the above paper remains this year (percentages are very different in non-general elections, btw), then we could apply a weight of .87 to likely voters (since 13% don't vote) and a weight of .55 to unlikely voters (since 55% do vote), which would predict that Romney would get 51% of the overall vote and Obama would get 49%, a result which pushes Gallup's result far closer to the average of other polls, where Romney has a slight lead nationally, and is slightly behind in the battleground states.

- Ravi Iyer