Problem retrieving data from Twitter

Cantor Loss shows Crowdsourcing, not Polling, is the Future of Prediction

Eric Cantor, the 2nd most powerful person in the House of Representatives, lost in the Republican Primary today to the relatively unknown Dave Brat. While others have focused on the historical nature of the loss, given Cantor’s position in his party, or on the political ramifications, I was most intrigued by the fact that polls conducted recently predicted Cantor would win by 34 points or 12 points.  In the end, Cantor lost by more than 10 points.

How did the polls get it so wrong?  In an age where people are used to blocking out web ads, fast forwarding through commercials, and screening their calls, using automated phone technology to ask people who they will vote for and assuming that you’ll get an unbiased sample (e.g. people who answer such polls don’t differ from those who do not answer automated polls) seems unwise.  The first banner ad got 44% clickthrough rates, but now banner ads are only clicked on by a small minority of internet users.  As response rates fall, bias is inevitable.

Pollsters may try to weight their polls and use new techniques to produce more perfect polls, but non-response bias will only get worse as consumers learn to block out more and more solicitations using technology.  On the other hand, a good crowdsourcing algorithm, such as the algorithm we use to produce Ranker lists, does not require the absence of bias.  Rather, such an algorithm will combine multiple sources of information, with the goal being to find sources of uncorrelated error.  In this case, polling data could have been combined with the GOP convention straw pollthe loss of one of his lieutenants in an earlier election, and the lack of support from Republican thought leaders, to form a better picture of the election possibilities as the non-response bias in regular polling is a different kind of bias than these other measurements likely have, and so aggregating these methods should produce a better answer.

This is easy to say in hindsight and it is doubtful that any crowdsourcing technique could have predicted Cantor’s loss, given the available data.  But more and more data is being produced and more and more bias is being introduced into traditional polling, such that this won’t always be the case, and I would predict that we will increasingly see less accurate polls and more accurate use of alternative methods to predict the future.  The arc of history is bending toward a world where intelligently combining lots of imperfect non-polling measurements are likely to yield a better answer about the future than one attempt to find the perfect poll.

- Ravi Iyer

Comments

comments

Men and Women Both Lie—But They Do It For Different Reasons

Reposted from this post on the Ranker Data Blog

lying-girlfriend-1085358-TwoByOne

We all tell white lies now and then (yes you do, don’t lie!) but did you know that men and women lie for different reasons? The data from our list of Things People Lie About All the Time shows a pattern that may hint at this difference.

The poll lists 49 common lies and asks respondents to vote “yes” if they’ve lied about that in the past 6 months or “no” if they have not. According to votes cast by over 350 people, women are more likely to lie about things that “keep the peace socially” while men are more likely to lie over matters of “self-preservation.”

On the list, women are 8 times more likely than men to lie about “being too swamped to hang out” and 4 times more likely to claim that their “phone died.” These results imply that women may be more likely to feel guilty about canceling on friends or having alone time.

In contrast, men were 2 times more likely to admit to saying things like “Oh yeah! That makes sense!” when they did not understand something and 5 times more likely to say, “No officer, I do not know why you pulled me over,” when, presumably, they did know why. These types of lies could point to men’s desire to show themselves in the best possible light and cover up wrongdoing.

Differences aside, both men and women voted similarly on many items on this list. In fact, the top 3 most popular lies were the same for both men and women.

The Top 3 Lies for BOTH Men and Women Are:

1. I’m Fine

2. I’m 5 Minutes Away

3. Yeah, I’m Listening.

Which goes to show that men and women may be able to see eye-to-eye after all… just as long as they don’t ask each other how they are doing, where they are and whether or not they are listening.

The post Men and Women Both Lie—But They Do It For Different Reasons appeared first on The Ranker.com Blog.

Go to Source

Comments

comments

Prediction: The Replication Crisis will be Solved by Market Forces, not Academics

Social science is in an interesting place, especially my home discipline of social psychology.  Traditionally, it has been practiced by scholars at academic institutions who are relatively unaffected by market forces, which meant that it didn’t really matter to people’s careers if what was discovered in social science was actually used, but rather that those discoveries were published in the right journals.  A few enterprising scholars used this fact to build lucrative careers by simply inventing discoveries that nobody was going to check.  Still others do things that are not fraud, but certainly increase the chances of finding positive results, that may or may not be generally true.  Indeed, I think every scholar engages in some form of this, including things that aren’t currently seen as biased like trying different stimuli/materials when your initial stimuli doesn’t work (I’ve done this).

There is no top-down cure for this.  We could (and perhaps should) strive for ways to make research more perfect, and worthy organizations are working on that.  But social science research is never perfect, always requiring a sample that is biased in some way and some compromise as far as control, ecological validity, and measurement, and perhaps this is where the metaphor of trying to emulate “hard” sciences like physics fails.  It doesn’t fail because the researchers are less intelligent or careful…indeed working with human subjects requires more ingenuity.  But rather it fails because while an experiment done on one rock most likely replicates on the next rock, human beings vary to much greater degrees.  We don’t all react the same to profound stimuli like the end of Romeo and Juliet, the election of a black President, or the sight of violence, so should we expect us all to react so predictably to subconscious primes or invented tasks?

I work as a data/social scientist because I believe in the utility and power of working using data on human behavior, but it is fundamentally different than experimenting on rocks or chemicals, in that findings are always probabilistic.   You can make a lot of money and do a lot of good things based on probability…but it is not the same kind of knowledge as the knowledge that allows my car to start in the morning, my refrigerator to run on electricity, and your computer to translate my words into pixels on your screen, where near-absolute predictability allows those items to function.

Probabilistic knowledge is better dealt with in markets, as compared to the current peer review journal system.  Social scientists actually study this.  The journal system is not well equipped to deal with things that are not black and white as all it’s constituent parts (what gets published, who gets authorship, who gets hired) are black and white.  Markets let people make bets, hedge their bets, and come to some probabilistic version of truth.

Social science, whether social scientists are a part of it or not, is moving toward being a market, as far more data on human thought, communication, and behavior is captured by the tech industry than is captured by academics, and the tech industry is fully responsive to market forces.  Statisticians and data scientists now publish far more knowledge than is available in scholarly journals.  Most tech companies do thousands of experiments each year and bet real money on the outcomes.

There is absolutely a place for the well designed academic study in this world, as there certainly are gaps in what is understood by industry processes.  But the insertion of market forces into social science is bound to change academics away from publication for publication sake and toward creating knowledge that is useful, as there will be real money at stake.  Publishing a paper will no longer be the end accomplishment, but rather the productive use of the knowledge gained by one’s research.  In that world, it won’t really matter what you think of the methods, statistics or claims of another researcher.  If you really don’t believe in a particular phenomenon, you can bet against it, or if you believe in a particular fact about human nature, you can bet on it.  And if there is no market for that bet, then maybe the question wasn’t that important to begin with.

- Ravi Iyer

Comments

comments

Predicting the Movie Box Office

Reposted from this post on the Ranker Data Blog

The North American market for films totaled about US$11,000 million in 2013, with over 1300 million admissions. The film industry is a big business that not even Ishtar, nor Jaws: The Revenge, nor even the 1989 Australian film “Houseboat Horror” manages to derail. (Check out Houseboat Horror next time you’re low on self-esteem, and need to be reminded there are many people in the world much less talented than you.)

Given the importance of the film industry, we were interested in using Ranker data to make predictions about box office grosses for different movies. The ranker list dealing with the Most Anticipated 2013 Films gave us some opinions — both in the form of re-ranked lists, and up and down votes — on which to base predictions. We used the same cognitive modeling approach previously applied to make Football (Soccer) World Cup predictions, trying to combine the wisdom of the ranker crowd.

Our basic results are shown in the figure below. The movies people had ranked are listed from the heavily anticipated Iron Man 3, Star Trek: Into Darkness, and Thor: The Dark World down to less anticipated films like Simon Killing, The Conjuring, and Alan Partridge: Alpha Papa. The voting information is shown in the middle panel, with the light bar showing the number of up-votes and the dark bar showing the number of down-votes for each movie. The ranking information is shown in the right panel, with the size of the circles showing how often each movie was placed in each ranking position by a user.

This analysis gives us an overall crowd rank order of the movies, but that is still a step away from making direct predictions about the number of dollars a movie will gross. To bridge this gap, we consulted historical data. The Box Office Mojo site provides movie gross totals for the top 100 movies each year for about the last 20 years. There is a fairly clear relationship between the ranking of a movie in a year, and the money it grosses. As the figure below shows, a few highest grossing movies return a lot more than the rest, following a “U-shaped” pattern that is often found in real-world statistics. If a movie is the 5th top grossing in a given year, for example, it grosses between about 100 and 300 million dollars. if it is the 50th highest grossing, it makes between about 10 and 80 million.

We used this historical relationship between ranking and dollars to map our predictions about ranking to predictions about dollars. The resulting predictions about the 2013 movies are shown below. These predictions are naturally uncertain, and so cover a range of possible values, for two reasons. We do not know exactly where the crowd believed they would finish in the ranking list, and we only know a range of possible historical grossed dollars for each rank. Our predictions acknowledge both of those sources of uncertainty, and the blue bars in the figure below show the region in which we predicted it was 95% likely to final outcome would lie. To assess our predictions, we looked up the answers (again at Box Office Mojo), and overlayed them as red crosses.

Many of our predictions are good, for both high grossing (Iron Man 3, Star Trek) and more modest grossing (Percy Jackson, Hansel and Gretel) movies. Forecasting social behavior, though, is very difficult, and we missed a few high grossing movies (Gravity) and over-estimated some relative flops (47 Ronin, Kick Ass 2). One interesting finding came from contrasting an analysis based on ranking and voting data with similar analyses based on just ranking or just voting. Combining both sorts of data led to more accurate predictions than using either alone.

We’re repeating this analysis for 2014, waiting for user re-ranks and votes for the Most Anticipated Films of 2014. The X-men and Hunger Games franchises are currently favored, but we’d love to incorporate your opinion. Just don’t up-vote Houseboat Horror.

The post Predicting the Movie Box Office appeared first on The Ranker.com Blog.

Go to Source

Comments

comments

Reducing Self-Interest Bias in Conflicts by Mitigating Disparities in Liking

Reposted from this post on the Civil Politics Blog

One of the most difficult things for all of us to overcome in any competitive situation is self-serving bias.  The below video explains it in an intuitive and entertaining way.  How many sports fans can be counted on to objectively view the decisions of referees?  Not many.  And similarly, how can we expect members of a group to objectively judge the fairness of actions of other group members?  Even those of us who take great pains to see the viewpoints of the other side are likely influenced by unconscious bias in service of our self-interest.

These same processes explain how both Jews and Palestinians have divergent historical narratives that they are completely convinced is the only view, how fiscal liberals and conservatives have completely opposite ideas about economic history, and how sports fans can be so convinced that they are routinely robbed by referees.  Opposing groups are often going to see facts in a way that conforms to their moral worldview (see research on and examples of moral coherence).

Self-serving bias may be ubiquitous, but there are still situations and circumstances that may reduce or exacerbate these tendencies.  Recently, at the 2014 conference for the Society for Personality and Social Psychology, I met Konrad Bocian who is investigating liking as a potential boundary condition.  Specifically (as is described in the below video), self-serving bias may occur only when it is done by people one has greater liking for.  In three studies, Bocian and colleagues measured moral judgments of rule-breaking behavior that benefited the judging party, and observed that feelings toward the perpetrator of the behavior were central to these moral judgments, even when the behaviors benefited the judging party.

This work relates to the Asteroids Club paradigm that is being pioneered by The Village Square, in that a central aspect of such meetings is to reduce the disparity in liking between members of one’s own group and members of opposing groups.  This hypothesis should be tested directly, but perhaps in moderating our feelings toward both our own groups and competing groups, we can mitigate some of the self-interest bias that exists in all conflicts and learn to disagree more productively.

- Ravi Iyer

Go to Source

Comments

comments

The Village Square helps partisans recognize common threats

Reposted from this post on the Civil Politics Blog

One of the more robust findings in social psychology is the idea that common goals reduce inter-group conflicts.  Several groups have recently taken this finding into the field, using Jonathan Haidt’s Asteroids Club model, including a dinner we co-hosted with the Nathan Cummings Foundation.  The group that has done the most with this concept is undoubtedly the Village Square, an organization that has put together a series of dinners where liberals learn about conservative concerns, and conservatives learn about liberal concerns, with the idea that people can come together, over food, to learn about issues that everyone should be concerned about.

Part of Civil Politics mission is to examine how research is used in practice and so we recently partnered with the Village Square to survey participants of a recent dinner where liberals learned about conservative concerns about the decline of individual moral behavior and conservatives learned about liberal concerns about moral corruption in politics (also see coverage in the Tallahassee Democrat).   We asked participants in the survey to agree or disagree with the following statements:

  • Liberals are generally good people.
  • Conservatives are generally good people
  • The decline of individual moral behavior is a serious issue that we should work together to correct.
  • The moral corruption of our political process through the influence of money is a serious issue that we should work together to correct.

 

The first thing we learned is that it is really hard to get people to answer survey questions with no payoff or incentive, and so only 10% of the approximately 150 people who attended completed the surveys.  As a result, the differences below are not statistically significant and consumers of traditional statistics would say that there is no difference.  A Bayesian approach (that I subscribe to) would say that this is relatively weak evidence.  With that caveat in mind, below are the survey results.

Village Square Asteroids Club Survey Results

It appears there were slight benefits as to how liberals and conservatives were perceived by the audience, with both groups being perceived as slightly more good.  However, the most important result is the last 2 bars, where, even in a case where participants already perceived the dual “asteroids” as serious, the event appears to have spurred some participants to take these threats even more seriously.  Research would indicate that forging a common bond should indeed lead to the possibility of greater inter-group cooperation.

That being said, this is indeed weak statistical evidence, given the small sample size and should be contextualized within the results of other Asteroid’s Club results.  Hopefully going forward, we’ll start to see a consistent pattern amongst events, such that sum of such weak evidence, combined with the results of lab studies, tells a consistent story.  If your organization is doing conflict resolution work (any conflict between groups will do, not just in the realm of politics) and would like to be part of that story, please do contact us and we would be happy to setup a similar survey for your event, to see if it does indeed bring people together, as well as to contribute ideas from our research.

- Ravi Iyer

 

Go to Source

Comments

comments

Cinco De Mayo and Tips for Marketing to the Hedonistic Pleasure Seeker

Cinco de Mayo (May 5th) is not a particularly important historical holiday.  However, it is an important holiday for both beer drinkers and beer companies, given that more than $600 million worth of beer is consumed on Cinco de Mayo, more than on The Super Bowl or St. Patrick’s Day.  This is one case where the market research supports my anecdotal experience: some people (and I fall into this group) just want an excuse to have a party.cincodemayo

Decades of social science research on values suggest that valuing pleasure (Hedonism) is indeed a stable personality trait that drives behavior.  While many wouldn’t elevate the pursuit of pleasure to the realm of higher order pursuits, there is no denying the fact that many individuals do indeed act as if the pursuit of pleasure is indeed a central life goal.  And many of those people will be drinking beer on Cinco de Mayo.  I’ve been working with Zenzi Communications to understand how to market to people for whom pleasure seeking is a central goal and below are 5 tips from ValueBase, our database of findings related to specific value types, for marketing the the hedonistic pleasure seeker:

  • Portray your product as an experience rather than a material item.  Encourage them to savor your product.
  • Emphasize the uniqueness of an experience.  Pleasure seekers enjoy being unique.
  • Leave out the “Green” messaging.  Pure pleasure seekers aren’t necessarily concerned with how environmental their consumption is.
  • Associate your product with the rich and famous.  Pleasure seekers appreciate the idea that what they consume is socially prestigious.
  • Aim for excitement!  Pleasure seekers are always up for an exciting new adventure.

Note that pleasure itself is not a single thing.  Recent research from Zenzi has found that people who seek moral pleasure or intellectual pleasure are actually quite different than those who seek sensual pleasure in many ways.  Contact Zenzi for more information.

- Ravi Iyer

ps.  looking for beer recommendation for Cinco de Mayo, look at Ranker’s beer graph.

 

Comments

comments

Gender and the Moral Psychology of Game of Thrones

Reposted from this post on the Ranker Data Blog

Most of my published academic work is in the field of moral psychology, where we study the moral reasoning behind judgments of right and wrong.  As I have previously argued, such study does not belong solely in the realm of university psychology labs, but also should be extended to the realm of “big data”, where online behavior is examined for convergence with what we see in the lab.  Ranker collects millions of user opinions each month on all sorts of topics, and one of them, where users rank the most uncomfortable moments in Game of Thrones, is actually very similar to psychology studies where we ask participants to rate the rightness or wrongness of various situations.

Amongst the situations to be voted on are:

  • Graphic Violence (Khaleesi Eats a Horse Heart, Execution of Eddard Stark)
  • Incest (Lannister Family Values, Theon Makes a Pass at Sister)
  • Sexual Violence (Danerys And Viserys, Jamie Rapes His Sister)
  • Homosexuality (Loras and Renly Shave and Scheme)

Men and women were equally likely to vote on items on this list (each gender averaged six votes per user), but women were twice as likely to be affected by sexual violence toward women, including Viserys’ lude treatment of his sister Danerys or The Red Wedding, which included the stabbing of a pregnant woman, than were men.  In contrast, men were made most uncomfortable by hints of homosexuality (Loras and Renly shaving each other’s chests), being seven times more likely to find this scene uncomfortable.  These patterns are convergent with research on mirror neurons, which indicate that people are most likely to be made uncomfortable by situations that threaten their self-identity, as well as accounts of women being driven to stop watching the show, due to the prevalence of depictions of violence against women.

Other patterns on this list also converged with previous research.  Americans, who may be less sensitive to violence due to its prevalence in American culture, were less affected by scenes such as the execution of Eddard Stark and Khaleesi eating a horse heart.  Southerners, who are more likely to be sensitive to purity concerns, were more affected by Petyr Baelish and Lord Varys’ discussion of perversity.

None of these findings are carefully controlled trials, so these patterns could have many explanations.  However, all research methods have flaws, and I would argue that it is the convergence of real world behavior with academic research that leads to true understanding.  Given Ranker’s new emphasis on Game of Thrones related content (like our Ranker of Thrones Facebook page if you’re a fan), more analyses of the repeated moral ambiguity in Game of Thrones are forthcoming and I would welcome new hypotheses to test.  What would you expect men/women to agree or disagree on?  Older vs. Younger fans?  West coasters vs. East Coasters?

- Ravi Iyer

Go to Source

Comments

comments

The Dark Side of Political Engagement. Unsolicited Advice for Brigade Media.

Reposted from this post on the Civil Politics Blog

I recently read with interest the news that several Silicon Valley entrepreneurs are attempting to “fix” political dialogue in this country through a well funded new venture called Brigade Media.  From this Time Magazine article:

With backing from early Facebook investor Sean Parker, Brigade Media LLC has already raised an impressive $9.3 million in funds to improve civic engagement from the federal level down to state and local politics

Guided by its scrappy startup ethic, Silicon Valley has disrupted entrenched industries from hotels to rental cars to pizza delivery, but a group of tech barons are raising the stakes with what may be their biggest challenge yet: American democracy.

As someone whose primary identity bridges the worlds of tech startups and using social science to bring people together, I’m excited to see where this effort leads.  However, I would caution tech entrepreneurs about confounding moral engagement with more productive disagreement, given research on the dark side of moral conviction.  Specifically, research in social psychology suggests that as people get more morally engaged with an issue, they also may become more rigid about those beliefs.  That is not to say that civic engagement is destined to decrease civility, but rather to highlight the potential pitfall in equating greater knowledge with more productive disagreement.  Research would suggest that efforts that promote relationships (e.g. see the work that our partners at The Village Square do) and/or promote super-ordinate goals (e.g. the Asteroids Club paradigm), at the same time that they promote civic engagement, may indeed lead to more productive civil debate.

- Ravi Iyer

Go to Source

Comments

comments

More Information does not necessarily lead to Civility

Reposted from this post on the Civil Politics Blog

A recent article by Ezra Klein at Vox.com eloquently makes an argument that we at CivilPolitics have also done a lot of research in support of – specifically, that if you want to affect many behaviors, you cannot just appeal to individuals’ sense of reason.  The article is well worth a complete read and is excerpted below, but the gist of it details a simple clear study by Dan Kahan and colleagues, showing that individuals who are good at math stop using their rational skills when the use of those skills would threaten their values.

How was this shown?  Consider the below table of results of a hypothetical study on whether a skin cream helps individuals with a rash.  Did the skin cream work well?  Simply scanning the numbers may give you the impression that the skin cream did well, as 225 is the highest number in the chart, yet if you look closer at the numbers, you’d find that the use of the skin cream is actually more likely to do harm than good, when compared to not doing anything at all.  However this kind of logical reasoning takes effort.

math-problem

 

Kahan’s work shows that we aren’t willing to make this kind of effort when the results would conflict with our values.  Specifically, when confronted with a ideologically charged political question (e.g. gun control) framed in the same terms, individual skill at math no longer predicts being good at solving such a problem .  Instead, one’s ideology was the main predictor and this was true for both liberals and conservatives.  From the article:

Presented with this problem a funny thing happened: how good subjects were at math stopped predicting how well they did on the test. Now it was ideology that drove the answers. Liberals were extremely good at solving the problem when doing so proved that gun-control legislation reduced crime. But when presented with the version of the problem that suggested gun control had failed, their math skills stopped mattering. They tended to get the problem wrong no matter how good they were at math. Conservatives exhibited the same pattern — just in reverse.

Being better at math didn’t just fail to help partisans converge on the right answer. It actually drove them further apart. Partisans with weak math skills were 25 percentage points likelier to get the answer right when it fit their ideology. Partisans with strong math skills were 45 percentage points likelier to get the answer right when it fit their ideology. The smarter the person is, the dumber politics can make them.

Consider how utterly insane that is: being better at math made partisans less likely to solve the problem correctly when solving the problem correctly meant betraying their political instincts. People weren’t reasoning to get the right answer; they were reasoning to get the answer that they wanted to be right.

If more information is not the solution to producing civility, than what is?  Our expertise at Civil Politics.org is in social psychology, which often concerns the subtle influences that can affect our non-rational side.  While we are still working on a comprehensive set of recommendations (check our blog for continuing progress and research), our social psychology page details a few simple principles that one can use in addition to providing information.  Specifically, getting people to like each other more can make them more open to opposing arguments.  Providing a non-oppositional framework also creates space that allows for more civil thoughts.  These themes also run through the work of organizations we work with, such as Living Room Conversations and The Village Square, which, consciously or not, effectively use social psychological principles in their work.  Whether you are more convinced by research in the lab, case studies, or a combination, the evidence is clear – more information, by itself, will not bring groups closer together.  To do so requires considering the many emotional and psychological motivations that we all have.

- Ravi Iyer

Go to Source

Comments

comments