Everybody Writes, Part One
December 6th, 2007“Everybody Votes” is a “channel” you can tune into on your Wii, Nintendo’s latest and greatest video game console. (If you can get one that is, but that’s a subject for another post.) “Everybody Votes” is the rare combination of two loves of mine: video games and data mining.
Basically, every couple of days you can put in a vote on some random topic Nintendo comes up with. You always have a choice between two distinct options, and you must choose one or the other. If you want to say “I don’t know”, the best you can do is choose not to vote on that topic. Typical questions are stuff like “Do you own a pet?” or “Do you like your name?”; as you might guess, most questions are about the minutiae of life. You also get to predict which choice will “win”, or have the majority of voters. That may or may not be the same as your personal vote on the topic.
Pretty simple stuff. Trouble is, Nintendo only gives you a limited amount of data at one time and doesn’t provide detailed statistics you can play with. I tried to write about “Everybody Votes” months ago and couldn’t find any data. But I looked again the other day and ended up finding one good source, the “Everybody Votes” blog on Blogspot. (Maybe it should’ve been called “Everybody Blogs”? Seems like a true enough statement these days…)
It was exactly what I was looking for. Not only does Tom (the guy who maintains the blog) provide screenshots of all the voting, but he also directly posts all the results as text in a very data-like format. Good stuff. Tom was also gracious enough to let me use his data for my post. Thanks Tom! (You’re a way better e-friend than that Myspace guy.)
Data in hand, I was now good to go. It appears that Tom’s been archiving “Everybody Votes” for about three months, for a total of 45 topics. Having voted in almost every poll, I can tell you that’s about 1/3 of the total polls they’ve done. (I’ve voted 138 times.) Ironically, I have exactly 45 losses in my predictions. (I don’t think they’re the same ones, but who knows?)
Nevertheless, 45 posts is more than enough data for me. So, what did I find out?
A lot, actually. So much that I have to split my analysis into 2 posts. This week I’m focusing on general trends; next week I’ll cover differences in voting by gender. (Hope you can handle this stuff for 2 weeks in a row, dear reader.)
One theory I had is that the first poll choice would be better and the option people would be most likely to select. The data suggest otherwise, however. On average, people select the first choice (the one on the left) 51.80% of the time and the second choice (the one on the right) 48.42% of the time.
Astute readers might point out that those two percentages add up to more than 100%. That’s a fair point - unless you’re Schroedinger’s Cat, you can’t pick both outcomes of a binary decision at once. But I triple-checked all my figures, and Tom’s percentages all added up too. My guess is that averages of both sides don’t add up because they’re not weighted by the number of voters. Some polls probably had more voters than others (especially as they went along), but I weight them all the same in my averages. However, this “overweights” polls with fewer voters and “underweights” polls with too many. If that’s the problem, though, it doesn’t seem to be a big deal because the percentages are fairly close. (Anyway, it’s good enough for government work, as they say.)
Anyway, the slight preference for the left choice could be noise, or it could be some kind of inborn preference for the left thing since we read from left to right (in the US, anyway) and so we see the left thing first. Or Nintendo could be putting the “obvious” choice on the left more often. My guess is that it’s noise.
How about prediction accuracy? Well, voters as a whole aren’t that bad at it. They predict what other people will vote 67.10% of the time, or about 17% better than chance. Since the theoretical maximum is 100% right and guessing is 50% right on average (especially since people have no clear preference for one side or the other), people are skillful enough to get about 1/3 more choices right than a guesser. However, since people are getting the other 2/3s wrong, there is plenty of room for improvement.
I’m no better. My prediction accuracy is 89 right and 45 wrong (a total of 134 predictions), or 66.42% (I didn’t give a prediction 4 times, I’m not sure why.) Despite all my knowledge about psychology, “Everybody Votes”, and polling, I can’t do any better than average. (Makes me wish I was from Lake Wobegone, where all the kids are above average.)
Here’s a graph showing prediction accuracy for all 45 polls graphically:
As you can tell by the title of that chart, people as a whole either predict modestly well (most of the time) or very poorly (occasionally). I wish I could say I’ve never been wrong when people predict that poorly, but I know it’s not true. I’ve definitely flubbed some predictions hardcore. (Who knew more people wear costumes for Halloween than not? Most people besides me, apparently.)
How much does the voting vary, then? As you might expect, there’s a fair amount of variance. The average spread, or the difference in percentages, between the two poll choices is 32.84%. (Though this can range from less than a percent to just over 50 percent.)
I had another theory that ties in with this. I speculated that the less spread there was (that is, the closer the vote), the worse people would predict. Turns out I was right. (Sadly I don’t think that will be an “Everybody Votes” question anytime soon.) Here’s a graph of prediction accuracy vs. voting spreads (by prediction accuracy in descending order):
It’s not perfect, but you can certainly see a correlation there. As prediction rates go down, so does the voting spread. Not too surprising, but it’s nice to see my theory confirmed by the data.
Another way to think about the data is prediction “skill” relative to chance vs. the “up or down” percentage. This is going to take a little explanation and a little math, so please bear with me. (I even subtract fractions. Channel your inner 4th grader if necessary. I know I did.)
“Skill” relative to chance is simply the prediction percentage minus 50 percent. That tells you how well you’re doing compared to guessing. Thus, if you predict correctly exactly half the time, your “skill” is 0 percent. (Note this can be negative if your prediction rate is lower than 50 percent. Although, in a way, even guessing wrong consistently is a form of prediction, I suppose. Just guess opposite!)
The “up or down” percentage is how much votes differ from a 50-50 percent split. So if 2/3s of people vote one way and 1/3 the other, the “up or down” would be 2/3 - 1/2 = 1/6, or 16.67% (the decimal repeats, so I rounded). If it’s 3/4s vs. 1/4, the “up or down” is 3/4 - 1/2 = 1/4, or 25%. All you’re basically doing is dividing the voting spread in half.
Thus, you’re comparing your skill, relative to a 50% baseline, against the voting variation, relative to a 50% baseline. It’s “apples to apples” and the measurements are of equal magnitude. The only weird thing is that your skill can be negative, while the “up or down” can only go as low as zero (if the vote is split exactly 50-50). Just goes to show you can do even worse than my theory allows. (Stupid Halloween costume wearers!)
Here, then, is a graph of prediction skill vs. the “up or down” percentage (by prediction accuracy in descending order):
You can see how closely predictions and voting spread track each other in that chart. (Though there’s still plenty of variation.) To bolster this claim, the average of prediction skill is 17.10% and the average “up and down” is 16.42%. That’s a correlation if I’ve ever seen one.
This jives well with what I already know about “Everybody Votes”. That is, voters assume other people think like them. They usually predict that people picked the same choice they themselves did. (I also know this from psychology - it’s called the false consensus effect.) So, if you predict that people vote the same as you, you’ll generally predict the best when the voting spread is large and people are more likely to vote with you. (Or you’ll get it spectacularly wrong from time to time, as you can see on the chart.)
Personally, I often predict that people will vote differently from me. I pretty much have to, since my “Everybody Votes” stats say that my “distance from popular opinion” is currently at 444 meters (and since it’s been 4 meters at one point, trust me when I say that’s a lot).
What kinds of questions are easy to predict the answer to, then? Questions like “Are you right-handed or left-handed?”, “Which kind of water tastes better? Tap water or Bottled water?”, or “Which is luckier? Four-leaf clover or Horseshoe?” Feel free to guess how those came out - they’re fairly easy to answer.
If only they were all that easy, though. Tough questions were stuff like “Which is more unlucky? Friday the 13th or a Broken Mirror?”, “Which kind of Halloween costume do you prefer? Scary costumes or Funny costumes?”, or “Are you afraid of speaking in front of groups?” (The latter had the worst prediction rate by far, at 23.80%, 16% worse than its next closest competitior.) Any guesses? Tune in for the answers (and some reflections on gender) next week!
(I’m going to post the spreadsheet I used, but not until next week so that you’ll have something to look forward to. If you look forward to spreadsheets, that is. And if you really need to know the answers to those questions before next Thursday, you can be a party pooper and dig through Tom’s blog.)
| | | del.icio.us |
December 14th, 2007 at 2:54 am
[…] a channel on the Nintendo Wii dedicated to voting on everyday topics. The first part is here. If you read any of those links they’ll bring you up to speed. You might be able to make […]
December 16th, 2007 at 2:04 pm
[…] they have done a two part analysis of the Everybody Votes Channel and it is really interesting. Part 1 talks about the general trends in the results of the polls and Part 2 talks about the differences […]
January 11th, 2008 at 1:51 am
[…] week’s post is part one of two on fortune cookies. As with my two part series on Everybody Votes, this week will be general observations on fortune cookies, while […]