Problem retrieving data from Twitter

Personality Types in Business: Conscientious CEOs & Open Technologists

Part of my job at Ranker is to talk to other companies about our data. While people often talk about how “big data” is revolutionizing everything, the reality of the data marketplace is that it still largely revolves around sales, marketing, and advertising. Huge infrastructures exist to make sure that the most optimal ad for the right product gets to the right person, leveraging as much data as possible. For example, I recently presented at a data conference at the Westin St. Francis in San Francisco, which meant that I spent some time on their website. For the past few weeks, long after the conference, I’ve been getting ads specifically for the Westin St. Francis on various websites. At some level, this is an impressive use of data, but at another level, it’s a failure, as I’m no longer in the market for a hotel room. The data to solve this problem is out there as someone could have tracked my visitation of the conference website, understood the date of the conference, and better understood my intent in visiting the Westin. However, this level of analysis doesn’t scale well for an ad that costs pennies, and so nobody does this level of behavioral targeting.

I bring up this story because I believe this illustrates a difference between how people who think of themselves as businesspeople and people who think of themselves as technologists often think. When talking about Ranker data, I often see this dichotomy. People who are more traditionally business minded want a clear business reason to use data, while people who think of themselves as technologists seem more open to trying to envision a world where data does all sorts of neat things that data should be used for. For example, I recently graphed opinions about beer, illustrating that Miller Lite drinkers were closer to Guinness drinkers than to Chimay drinkers. As a technologist, I’m certain that a world will soon exist where bartenders can use data about me and others like me (e.g. the beer graph), to recommend a beer. I don’t worry as much about the immediate path from the conception of such data to monetization. I know that the beer graph should exist and I’m happy to help contribute to it, confident of my vision of the future.

This division between people who think like businesspeople and people who think like technologists is important for anyone who does business development or business to business sales, especially for those of us in the technology world where the lines are often blurry. Mark Zuckerberg is a CEO, but clearly he thinks like a technologist. My guess is that a lot of the CTOs of big companies actually think more like businesspeople than technologists. If I were trying to sell Mark Zuckerberg on something, I would try to sell him on how whatever I was offering could make a huge difference to something he cared about. I would sell the dream. But if I were selling a more traditional businessperson, I would try to sell the benefits versus the costs. I would have a detailed plan and sell the details.

I actually have a bit of data from to support this assertion. We have started collecting data on visitors’ professions and below I compare businesspeople to technologists on two of the Big Five personality dimensions that are said to underlie much of personality: Conscientiousness and Openness to Experience. As you can see, businesspeople are more conscientious (detail oriented, fastidious, responsible), while technologists score higher on openness which is indicative of enjoying exploring new ideas and thinking of new possibilities.

Personality Types in Business

The reality is that every business needs a balance between those who are detail oriented and precise (Conscientious) and those who think about a vision for the future (Openness to Experience). Often, technologists who start a company will eventually hire professional businesspeople who provide this balance (e.g. Sheryl Sandberg or Eric Schmidt). Clearly, the best sales pitch will be both detailed and forward thinking. However, if you’re talking to someone and have limited time and attention, considering whether you are speaking to someone who is more of a businessperson or more of a technologist may give you better insight into how to frame your pitch.

- Ravi Iyer

ps. Crossposted on Zenzi Communications‘ blog here, which is using a data driven approach to improving communications strategies.

The Moral Foundations of Environmentalists

I was recently asked about the Moral Foundations scores of those who are more concerned about the environment and so I analyzed the 15,522 individuals who took the Moral Foundations Scale on and also answered a question on the Schwartz Values Scale concerning how much of a guiding principle of their life it was to “Protect the Environment”.  I limited this analysis to those who placed themselves on the liberal-conservative spectrum, so that I could also control both for ideology and extremity of ideology, to some degree.  The results (beta weights controlling for other variables) of the regression analyses, predicting a desire to “Protect the Environment”, are below.

Moral Foundations of Environmentalism

My initial intuition was that ideology would be the greatest predictor, given how political the issue has become, but it appears that the Care/Harm foundation actually predicts as much unique variance as ideological identification.  From an intuitionist standpoint, this makes sense as the specific care you feel for Polar Bears may drive one’s values more than more abstract concerns about the ocean’s water level, similar to the way that charities appeal to emotions with specific cases of need as opposed to statistics.  Still a great deal of variance is indeed predicted by which ideological team you are on.

Also interesting to me was the significant, but small, negative relationship between ingroup loyalty and attitudes toward the environment.   The item I used from the Schwartz Values scale is part of a subscale designed to measure Universalism, which relates to Peter Singer’s idea that we should expand our moral circles.  While it is certainly possible to care both about one’s smaller circle/family and one’s larger circle/animals/trees, there is some tension there, especially in a world with limited resources where environmental choices that benefit the world at large, may negatively impact one’s local community.

There are certainly limitations to these results taken from a particular sample, so take them with a grain of salt.  And there remains a healthy debate about which moral concerns are more central, so there certainly are moral concerns that may predict environmental attitudes that are not measured here.  Still, these results converge well with what we see in the world.  Environmentalists tend to be liberals who are particularly concerned about the welfare of distant others, perhaps expanding their moral circle to include animals, oceans, and trees.
- Ravi Iyer

Which undecided voters use the internet to figure out who they should vote for?

In 2008, I co-founded as a way to help people with the question “Who should I vote for?”  In 2008, it served over 500,000 people, but we didn’t get any demographic information at the time, so, while valuable, I couldn’t answer many of the questions I wanted to answer about the use of “candidate calculators”, which is a name sometimes given to sites that allow you to enter your political opinions into a website, which then attempts to match your opinions to those of political candidates.  In 2012, I added a few optional questions to the end of the quiz that asked the age, gender, political ideology, and planned candidate choice of quiz takers.

Right now, we rank #2 or #3 (I’ve seen both) for the search query “who should I vote for” on Google and according to Google, about 15,000 people have searched for that query over the past 30 days, with about 5000 clicking on  Some number of people do not fill out our surveys (25% bounce rate) and of those, only 30% or so fill out our optional demographics questions.  Browser referral information isn’t always sent, so I can only identify 470 who definitely typed in “who should I vote for” into Google to come to our site during the 2012 election, the bulk of which occurred in the last 30 days.  Still, I think it’s perhaps indicative of the kind of person who searches the internet for voter information.

Who is this person?  The average age was 30.3 years old (SD = 11.1), with people as young as 12 and as old as 87 taking the survey.  56% of quiz takers were male.  Judging by the below charts, the average person who asks Google who they should vote for really is likely to be undecided and moderate.

What do these voters care about?  In order to eliminate the effects of the liberals and conservatives, I looked just at the 201 people who said they were moderate or apolitical.  Here is the list of issues they care about in descending order of importance.

Issue Importance

And here are their stances on these issues, with questions they agreed to listed first, and questions they disagreed with listed last.  Note that these questions were asked on a 7 point scale with 1 = strongly disagree, 4 = in the middle, and 7 =  strongly agree.

Statement Agreement

Statement Agreement

What can we conclude from these analyses?  It seems like the kinds of people who are asking for help on the internet are people who might be classified as populists.  They appear to be mainly younger men, who want compromise in government, favor liberal policies like higher taxes on the wealthy, higher spending on education, and more corporate regulation, but also favor conservative policies like stricter immigration enforcement and stricter controls on government spending.  Of course, perhaps taking the average of these undecided voters obscures differences among these voters.  Also, these results are likely to generalize best to the types of individuals who are actively using the internet to figure out who to vote for, since our sample all typed in “Who should I vote for?” into Google and then took the VoteHelp quiz.  On one promising note for these analyses, these results do seem to converge with the media’s depiction of the voters who both campaigns appear to be trying to woo right now.

- Ravi Iyer

Will Mitt Romney appeal to empathizers or systemizers in his Republican National Convention speech?

The Republican National Convention is going to take place this week and one of the stated goals of many republicans is to “humanize” Mitt Romney. It reminded me of this graph that I pulled from our database which looks at systemizing vs. empathizing scores.  Based on work by Simon Baron-Cohen, the measure concerns how much one likes to analyze and construct systems as a way of understanding the world (e.g. being fascinated by how machines work) versus trying to understand social situations and empathize with others (e.g. I am quick to spot when someone in a group is feeling awkward and uncomfortable.).  Men (in general) tend to systemize, while women tend to empathize and this difference tracks rates of autism (Baron-Cohen’s main line of research), which strikes 4 males for every 1 female.  Men also tend to support Romney vs. Obama.

This graph shows the correlation between favorability ratings of potential 2008 presidential candidates and the difference between systemizing and empathizing scores for those candidates’ supporters.

Based on our libertarians research, we would have expected Ron Paul supporters to have the highest systemizing vs. empathizing scores and certainly his supporters do have a positive, and relatively high correlation.  It is similarly unsurprising that Hillary Clinton’s supporters in 2008 tended to be empathizers, or that Democrats generally tend to attract empathizers, rather than systemizers.  What surprised me, however, was that Mitt Romney’s supporters appear to have the highest systemizing vs. empathizing difference.  Does this reflect something intrinsic about Mitt Romney, or at least his image?  After watching some of the Sunday shows today, I think so.

Consider this quote from ABC’s This Week, by George Will, a conservative who observed that “with most politicians, the problem is their inauthenticity.  His (Romney’s) problem is that he is authentically what he is…he has a low emotional metabolism.  That’s who he is.  He can’t turn to the country and say I feel your pain because the pain isn’t his.  It’s other people’s.  What he can say is that I can fix your pain and that should be good enough for most people, unless we are electing a talk show host”.

Mike Huckabee said something similar on Fox News Sunday about likability being less important than technical skill.  These are perhaps inherent admissions by some of Mitt Romney’s supporters that his strength is in appealing to systemizers, and therefore, they would like frame the debate in those terms.  It will be interesting to see whether Mitt Romney aims his Republican National Convention nomination acceptance speech at empathizers or at systemizers.

-Ravi Iyer

Where to live? Liberal, conservative, & libertarian criteria differ.

In a line of research led by Matt Motyl, at the University of Virginia, we’ve been exploring ideological differences in preferences for where one lives.  This project is informed by a few ideas already out there.

Given these trends, we would expect liberals, conservatives, and libertarians to differ on what traits are most important in choosing a city to live in.  To test this, we asked participants to allocate 100 importance points to the 10 (out of  46) most important traits that they would use to judge a city.  The idea was to force people to make choices about what is and what is not important as most all of these traits are desirable.  The results, based on over 2000 visitors, largely follow common sense and are shown below with the traits preferred by liberals at the top and by conservatives at the bottom.  For the statistically minded among you, all correlations of .05 or higher are statistically significant.

Where to Live preferences for liberals, conservatives, and libertarians

Perhaps more interesting are the average number of points allocated by liberals, conservatives, moderates, and libertarians to each of these traits.  There is actually a great deal of consensus as to what is important (clean air/water, safety, job opportunities, medical care) even as there are differences (public transportation, family friendly, religiosity).  Also interesting is to note aspects of cities for which libertarians score highest (not too noisy, scientific community, many atheists), which dovetails well with our other research on libertarians.

Average points allocated by ideological group:

Where to Live Points Allocated

There are important plusses and minuses of using non-representative samples.  However, these results generally conform to popular wisdom about these groups, so while the means may differ in the general population, the overall patterns seem likely to generalize.  As with much of our research, the goal isn’t to determine which way of being or which city type is best, but rather to help people more explicitly make choices that may align with their value orientation.  I’m hopeful that the above lists will prove generative when people search the internet for ideas about where to live, a search which apparently is getting more and more common, according to Google Trends.

Where To Live Google Trends

- Ravi Iyer

Psychological Differences between Cell Phone and Landline users presents a challenge for polls

When the NY Times or Gallup reports that Obama or Romney has a lead in the polls, how do they know this?  Typically, they pay people to randomly call people and they extrapolate from this sample, using established statistical methods, to make generalizations to the population.  Some groups won’t respond, especially young adults who often have cellphones and screen their calls.  Many people I know are like this.

Polling guru, Nate Silver, has written about this issue extensively, and Pew has researched it as well.   Cell phone users tend to be younger and more liberal.  Pollsters are used to correcting for such selective non-response (e.g. men selectively non-respond more than women) by weighting their answers.  However, this critically relies on having a variable that you can use to do this weighting.  If cellphone users simply differ on demographic dimensions, weighting should work, but if they differ on other dimensions such as Big 5 personality traits or values, then pollsters will be unable to weight their data.

Do cell phone users differ from landline users on psychological dimensions? The answer is fairly common sense as this is an issue that we all have lots of anecdotal data on.  Of course they do.  The below chart compares cellphone users to landline users based on visitors to who answered a question about their phone usage, with traits related to landline use at the top and traits predicting cellphone use at the bottom.

Psychology of Cell Phone Users

Cellphone users value stimulation, achievement, and hedonism more.  They value tradition, conformity, and security less.  They are less conscientious, more liberal (especially on social issues), and are younger.  Some of these variables are things that pollsters can address by weighting their results (e.g. youth and liberalism), but other variables are things that pollsters do not measure and therefore cannot directly weight for.

Since some of these things vary by ideology, gender and age as well, we can statistically control for these factors and see if we get fewer significant predictors of cellphone usage.  Valuing stimulation and achievement are the remaining significant predictors with valuing tradition and being socially conservative as marginally significant predictors.  Other psychological variables such as being conscientiousness and valuing hedonism are accounted for by controlling for factors that pollsters likely can weight for.  As such, perhaps these psychological variables are less problematic.  It is worth noting that valuing stimulation remains by far the best predictor of cellphone usage (after age) in regression analyses controlling for demographic variables (beta = .13, p<.001).

The sample is not a representative sample, but I think that might be better in this case.  Trying to measure characteristics of people who use cellphones, which I would assume correlates with screening calls and generally being less responsive to surveys, might be better done using non-phone means so that your measurement interacts less with what you are measuring.  The educated, internet savvy users who tend to answer yourmorals surveys are exactly the kind of people you might want to examine and be unlikely to poll via phone.  Further, we aren’t interested in whether the overall population has differences between cellphone users and landline users.  That could be a function of youth (the biggest predictor here).  Rather, we are interested in whether people who have the exact same demographic characteristics and vary only in terms of their cell phone usage may differ in meaningful ways as it is this variance that would confound pollsters.  Using a particular non-representative sample can actually be better for answering questions about the relationship between variables, as certain differences are naturally controlled for with the whole sample being generally internet savvy, educated, and white.  But certainly these findings (like all social science) need to be replicated by others in other datasets to have more confidence.

The take home message?  As noted by Pew and Nate Silver, polls will have to have cellphone samples in order to avoid bias that likely skews against liberal candidates.  Second, if my intuition that heavy cell phone users are unlikely to respond regardless is correct, then even pollsters that poll cellphones may have to start thinking about weighting for non-traditional variables that are a proxy for these psychological variables that predict non-response.  Silver suggests “urban/rural status, technology usage, or perhaps even media consumption habits“.  Third, the psychological profile of cellphone users (seeking novelty, being socially liberal and not valuing tradition) suggests that polls might exhibit more bias on social issues such as gay marriage, and other issues which could reasonably be said to correlate with being a novelty seeker.  These effects aren’t big, but in a world where a few percentage points is big news, they are worth considering when digesting poll results.

- Ravi Iyer

Empathizing vs. Systemizing – A Book Review of Tattoos On The Heart

I recently read this article from Fast Company about Father Greg Boyle’s work at Homeboy Industries, and just like every other time I’ve encountered stories of this work, it ended with me in tears.  It reminded me that I’ve been meaning to write about Tattoos On The Heart, which just might be my favorite book ever.  It certainly is the most moving book I’ve ever read.

Since this is a blog that is largely about psychology, I’d like to frame my discussion of the book in terms of one of my favorite psychological theories of personality, Simon Baron-Cohen’s Empathizing-Systemizing distinction.  Father Boyle is a great empathizer, who seems to “enjoy caring for other people”, is able to “predict how someone will feel”, and knows “what to do in a social situation” (quotes are from Baron-Cohen’s scale).  In contrast, he is a fairly mediocre systemizer (e.g. reading “legal documents very carefully”), if we are to infer that trait from the finance side of Homeboy Industries depicted in Fast Company.  Luckily, he now has help.  This empathizing dimension relates to the two things that I feel are most powerful about Father Boyle.  His ability to forgive and his ability to tell stories.  From the book:

We had lots of enemies in those early days, folks who felt that assisting gang members somehow cosigned on their bad behavior.  Hate mail, death threats, and bomb threats were common…From my office once, I heard a homegirl answer the phone, and say to the caller, “Go ahead and bring that bomb, mutha fucka.  We’re ready for your ass.”…”Uh, Kiddo, um,” I tell her, “Maybe we should just say ‘Have a nice day and God bless you.’”

Some of the gang members have done terrible things, but one of his favorite things to say to those whom most of society would rather ignore is that “you are so much more than the worst thing that you have done.”  In the Fast Company article, they give money to a woman who punched their receptionist in the face.  Sometimes the generosity seems so without limits as to be insane, yet for these youth who have no fear of prison or death, it seems hard to imagine anything but unconditional love being their salvation.  In some ways, Father Greg is giving these youth the unconditional love that many of us take for granted from our parents.

Our data tells a similar story about the characteristics of empathizers.  Empathizers (the blue line) in our dataset, tend to forgive others (as measured by questions like being “understanding of others for the mistakes they’ve made”).

As well, empathizers, in our dataset, also tend to enjoy stories (r=.17, p<.001, N=495), and the second trait that makes Father Boyle unique is his ability to tell stories.  Stories are a way for human beings to communicate not just information, but the feelings that go along with that information.  Indeed, the most common measure of empathy used in psychology, the Interpersonal Reactivity Index, uses items like “I really get involved with the feelings of the characters in a novel” and “Becoming extremely involved in a good book or movie is somewhat rare for me” (reverse scored) to measure empathy.  Stories are powerful things.  From the introduction of the book:

I have all these stories and parables locked away in the “Public Storage” of my brain, and I have long wanted to find a permanent home for them.  The usual “containers” for these stories are my homilies at Mass in the twenty-five detention centers where I celebrate the Eucharist…After Mass once, at one of these probation camps, a homie grabbed both my hands and looked me in the eye.  ”This is my last Mass at camp.  I go home on Monday.  I’m gonna miss your stories.  You tell good stories.  And I hope….I never have to hear your stories again.”

Father Boyle’s stories really are good and show the polish of years of curation.  They transform me every time I read them, reminding me that while justice may feel good, kindness is far more powerful.

If there is a fundamental challenge within these stories, it is simply to change our lurking suspicion that some lives matter less than other lives.

Food for thought.  Please do read the book and I’ll be quite shocked if you can read the stories in the book without being similarly moved.  I can’t recommend it enough.

- Ravi Iyer

Big 5 Personality Traits of would-be Owners vs. Renters

I feel as if sometime in the early 2000s, society collectively decided that it was better to own a home than rent.  Property values went up and it seemed like people were willing to go to great personal difficulty simply for the sake of being an owner.  It probably didn’t hurt that property values kept going up.  Still, I never felt a strong urge to own and the prospect seemed more like a burden (fixing your own things, having trouble being able to move) than a blessing.  Of course, that may say more about my personality than about owning or renting.

I thought I’d examine the Big 5 personality traits of people who think owning is “better” (e.g believing that home ownership is important to happiness) vs. those who prefer renting (e.g. believing that renting provides significant advantages compared to owning a home) using ~800 people who answered these questions at  I had 7 questions about owning vs. renting (alpha = .87).  The Big 5 personality traits are 5 personality dimensions that are deemed most parsimoniously able to characterize people.  The dimensions are Agreeableness (e.g. how well do you want to get along with others), Conscientiousness (e.g. how detail oriented and tidy are you), Extraversion (e.g. how outgoing are you), Neuroticism (e.g. how tense are you), and Openness to Experience (e.g. how much do you seek out new experiences).

Predictably, people who prefer owning a home vs. renting are more conscientious (r = .08, p=.016) and less open to new experiences (r = -.08, p=.03), but the differences are quite small.

Would-be owners are more conscientious and less open to new experiences.

People who want to be owners also also tend to be more conservative (r=.18, p<.001), older (r=.13, p<.001), and tend to prefer buying material things rather than experiences (r=.13, p<.001).  Interestingly, there was no relationship to self described social status or gender.  Obviously many of these relationships are small, but they certainly are as I would predict, with perhaps the exception of the lack of relationship with wealth and gender (my guess would have been that women and wealthier people would prefer home ownership).

Got any interesting hypotheses relating to the personalities of those who prefer renting vs. owning?  I’d happily try them.  I’m eager to examing values with regard to owning/renting next.

- Ravi Iyer

Relative vs. Absolute Good Choices for Liberals, Conservatives, and Libertarians

Awhile ago, I read about a survey given to Harvard Medical school students about whether they would prefer to live in a world where they had a higher absolute amount of some beneficial good or a higher relative amount.  For example, participants had a choice of living in a world where they make $100,000 and everyone else makes $200,000 (absolutely better) or one where they make $50,000 and everyone else makes $25,000 (relatively better), explicitly assuming buying power remains the same.  The same types of choices were made for IQ, education, vacation time, attractiveness, and other goods, with the choice being between having more of something (absolute) or having more than other people (relative).  The survey results often generate a lot of discussion, in my experience, as people are intrigued by the idea that lots of people would give up money, just to be better than others.  In truth, other studies have shown that almost everyone cares about relative concerns, just perhaps in different circumstances.

I ran the same survey at, and the results are similar to the original study, with some important differences (see graph below).  Importantly, the % of people who chose a world of relative income was smaller than in the original study, where 50% of participants chose relative position.  Perhaps people at Harvard are simply more competitive?  Mean scores are quite variable in different non-representative samples, so I wouldn’t put much stock in them, but perhaps more interesting is that the relationship between variables replicates.  Our results converge with the idea that some goods are more positional than others.  Specifically, the same things that people thought were more appropriate to think of in relative terms in the original study (praise and attractiveness) were thought to be relative in our sample, with vacation time being the least relative good.  The graph below shows questions in rough decreasing order of concern about relative position.

relative vs. absolution goods in liberals, conservatives, and libertarians

Our data suggests that some people think of things as more relative than others.  Cronbach’s alpha for the items in the graph was .80, meaning that answers positively correlate and it is reasonable to think of answers to these diverse questions as all representing some general underlying preference for relative or absolute position.

Interestingly, it appears that conservatives care more about relative position compared to both liberals and libertarians.  Perhaps this converges with the idea that conservatives have a more competitive orientation, leading to positive beliefs about competitive markets and competitive sports, both of which are found in our data as well.

The current data is based on 5,795 participants (3,559 liberals, 632 conservatives, 569 libertarians, and 1,035 others) who took this survey.  This means that aside from political orientation, we could look at other factors that are associated with preference for relative or absolute goods.  For example, concern for positional goods is negatively correlated with Big 5-Agreeableness (r=-.13, p<.001), Openness to Experience (r=-.09, p<.001), and positively correlated with Neuroticism (r=.07, p<.001).  These are very modest correlations made significant by the sample size that took both measures (3,844).  If other people have ideas for personality variables that may explain why some people prefer relative vs. absolute goods, please leave a comment with your ideas.

- Ravi Iyer

Liberals place more value on being funny than conservatives and libertarians.

I’ve been watching a lot of comedy central lately and have been wondering why there does not appear to be a conservative equivalent, just as there is no popular liberal equivalent to conservative AM talk radio.  Perhaps liberals value being funny more than conservatives?

To test this idea, I thought I’d look at the data from the Good Self Scale from  In it, participants are asked how important it is to have various traits, and one of them happens to be “funny”.  If you look at the below graph, you’ll see that liberals do indeed place a tiny bit more value on being funny, compared to others (p<.01 comparing liberals to non-liberals).

It is important to note that this does not mean that liberals are indeed funnier, but rather that they place a value on being funny.  The results seem plausible given that the rest of the results conform to previous research (e.g. conservatives care about loyalty more and care about being more responsible).  Some observations:

In all, these are fair descriptions of these ideological groups, and given that the other relationships are reasonable, I would conclude that it’s also reasonable to say that liberals likely do place more value on being funny than other ideological groups.  Whether they succeed or not is another question.

- Ravi Iyer