“Predictions Are Often Wrong, Especially About the Future”
March 7th, 2008I’m not quite sure who’s responsible for the quote that’s the title of this post, but whoever said it, I agree. I’m not even sure where I first read the quote, but I my best guess is that it was in one of Nassim Taleb’s books, which I blogged about a while back. But just because we’re bad about predicting the future, does that mean we can’t find a better way to do so?
Well, some events are easy to predict, at least. These are events that Taleb says belong to “Mediocristan”, a fictional place where everything happens in small, predictable spurts. This includes predictions about things like people’s average height and weight. Most people aren’t going to be 7 foot tall. (Unless you live in “Basketball-istan”.) I can predict that. “Mediocristan” events such as these often follow linear trends and “bell curve” distributions. This is another way of saying that there are experts in the average weight and height of people.
By contrast, events belonging to “Extremistan” are defined by a few, sudden, unpredictable events Taleb calls “Black Swans”. Financial market movements belong to “Extremistan”, and so do events like environmental disasters and the next American Idol winner. These events are swift, incredibly volatile, and impossible to predict. (If they were possible to predict, they wouldn’t be “Black Swans”, by definition. And American Idol would probably be a lot less interesting.)
Lately, however, there’s been another kind of event I think belongs to Extremistan - elections. For example, pundits were spectacularly wrong about the Democratic primary in New Hampshire.
Of course, this isn’t the first time the polls have been off. In the past, pollsters overestimated Howard Dean’s chances of winning the Democratic nomination, in addition to John Kerry’s chances of becoming president. You can chalk up Dean and Kerry to close elections, bad samples, and lying responders, but the New Hampshire polls were off by perhaps 20 points. That’s as big a failure as you could possibly expect in an election poll. It seems that Clinton’s New Hampshire victory was a Black Swan, at least to pollsters and the media.
If pollsters can get it wildly wrong on occasion, where can we turn for more reliable data? Well, we could always ask Lady Luck and try prediction markets, like Intrade. If you’re not familiar with the site, Intrade is a prediction market site where people bet on the probability of a (usually binary) event either happening or not happening. Intrade’s biggest and most highly traded prediction markets are all political. All you have to do is decide to buy or sell a particular contract (like, say, Hillary Clinton winning New Hampshire) and find someone willing to be the “other side” to your trade, and you’re good.
Traders make up the entire market, as Intrade does not take any positions one way or the other (they take a commission instead). Contracts can range in price from 0 (everyone thinks the contract will not pay off) to 100 (everyone thinks it will). You can sell at any time as long as you can find a buyer, but if you hold on to contract to expiration, the contract will either end at 0 (it didn’t happen) or 100 (it did). And thus, you’ll either lose all your money (including your margin to cover downswings) or make the difference between what you paid for the contract and what it ended at, minus a small expiration fee. (If you want some more background on the subject, there’s an excellent scanned Economist article here.)
So how did the prediction markets do in New Hampshire? Sadly, until the day of the New Hampshire Democratic primary, Intrade’s political markets were off the mark too. Still, it may not have been Intrade’s fault that the markets were so off-base for so long, since there may have been other factors at work. Plus, Intrade’s markets incorporate new information faster than just about anywhere else.
In fact, like it says in the article I just linked, rather than looking into the future, prediction markets are more like “immediate-past” markets. From my extensive watching at Intrade, I’m inclined to agree. As I said earlier, prediction markets, like financial markets, seem exceptionally good at incorporating the latest information quickly, since it’s in the financial interest of the participants to do so. In that vein, I often find Intrade to be a better, more timely, and relatively unbiased source of political information compared to most conventional news sites. When you have to put your money where your mouth is, I think you’re a little less inclined to mouth off. (Unless you’re buying attack ads, I guess.)
Still, prediction markets are not without their critics. I’ve seen contracts that were sure losers (like betting that Dennis Kucinich will win the Democratic nomination) that were propped up by supporters or not taken advantage of because of a lack of liquidity (i.e., someone on the other end of the transaction). Sometimes, even the fees Intrade charges get in the way of otherwise profitable contracts.
Probably one of the biggest problems facing predictions markets, though, is the fact that they’re illegal to bet on in the U.S., since they’re covered in the new law banning internet gambling. This was a particular problem for me, since I wanted to try out some prediction markets for myself without having to take any awkward calls from my bank, like a friend of mine did that’s into online poker.
Luckily, Intrade has set up a site where you can gamble with play money (which several other sites have set up as well). Considering my previous experiences with poker, I had every intention of betting real money there, but with the new law I figured play money would have to do. Anyway, if you sign up at play money site, they give you $5,000 in play money to speculate with. It’s not a bad deal really - Monopoly only starts you out with $1,500, after all, and the games of it I used to play took days. (Curse you, Free Parking!)
However, the biggest reason I signed up to Intrade was actually to test a political theory of mine. I knew Intrade could be wildly off about political matters. But was that because of biases, liquidity, and others issues with the market, or because of “Black Swans”? That is, could I successfully predict where pundits and prediction markets failed? I wasn’t sure, but I wanted to at least give it a shot. (And maybe win a little funny money, too. Finally, revenge for all those times I landed on Boardwalk!)
My trading theory was simple - only bet according to demographics. I took a look at the demographic makeup of several states with upcoming primaries, and I bet accordingly. The idea here was that I bet on the candidate that shared the most demographics with the most voters there. For example, since a large proportion of Georgia is black, young, and religious like Obama, I bet on him to win the Democratic primary in GA. Of course, that was a pretty lopsided bet on a favorite, so winning the contract was a piece of cake. But my theory would also have explained Clinton’s victory in New Hampshire since she had a clear demographic advantage. Anyway, here’s a list of all the contracts I bet on to “win” in the various primaries (I held all of the contracts until expiration, since despite an “expiration fee” it usually worked out in my favor):
- Clinton in California
- Giuliani and McCain in Florida (the Giuliani bet was a bit of a lark, so sue me)
- Obama in Georgia
- Clinton in Texas
Despite some of those contests being quite close, I ended up 4 for 4 (not counting my small Giuliani bet) and up about $2,800 in play money. (By the way, in case you’re wondering, the play market usually closely tracks the real market.) I used the same strategy in each situation. For example, when I bet on McCain, I knew there were a lot of older white folks in Florida that were religious, but not Mormon. (People here often morbidly joke that Florida should be called “God’s Waiting Room”.) I figured the closest demographic match for most voters would be McCain over Romney, and I was right, even though the vote was within a couple percentage points.
The same went for Clinton in Texas. I knew that a lot of Texas voters were older, white, working-class people that would identify with her. Even if every voter didn’t fit that profile, I knew enough people would share enough pieces of it to be useful. So, despite the fact that my contract was negative for much of the time I held it, I ended up winning in the end. (That vote was also within 2-3 percentage points.)
I should mention that I definitely didn’t wait until the last minute to buy these contracts. I usually purchased them 2-4 weeks in advance. At least two of them were near 50 when I bought them (meaning many people thought the contract could go either way). And if you look at contracts like Clinton in Texas, you’d see that it was usually strongly against her up until voting day. In the end, my strategy had prevailed where prediction markets had mostly failed. (And probably pollsters too, since they said the Texas Democratic race was too close to call by the day of the primary.)
Of course, I should mention some reservations. I’m well aware that, despite wanting to believe in my newfound oracular and funny-money prowess, my 4 bets do not constitute anything like irrefutable evidence for my claim. And I’m conscious of the statistical regress and ludic fallacies. The “statistical regress fallacy” warns against seeing a pattern from analyzing past data (especially a small sample like this) and the “ludic fallacy” warns against using games and models (like prediction markets) to predict real-world events (like political primaries).
Still, just like I believe that the economy has an oversized effect on presidential approval ratings (”it’s the economy, stupid”), I also believe demographics has an oversized effect on elections. And I think my small sample bears this out somewhat.
Maybe political events like Hillary’s performance in New Hampshire aren’t Black Swans after all. Maybe they’re just based on demographic trends overlooked by the sampling methods pollsters choose and the models of those who bet in prediction markets. Only time will tell.
With my theory in mind, then, who do I think will win in November’s Presidential election? If it’s McCain vs. Obama, it’s a tough call. About the same proportion of the country is black (good for Obama) or over age 65 (good for McCain). And while more voters will be near Obama’s age, more voters will also be white like McCain. Obama is probably more religious than McCain, which could increase his appeal since a lot of Americans are religious, but many of the voters close to Obama in age are less likely to be religious, so this may not matter as much.
In the end, I don’t think the demographics are clear in this proposed contest. Based on my theory, I think it would probably be close. If I had to guess, I’d take the underdog bet and say McCain, since there are more veterans and older or white Republicans/Independents (those more likely to vote for McCain) than there are young or black Democrats/Independents (those likely to vote for Obama).
If it’s McCain vs. Clinton, it seems like an easier choice. There are more women in U.S. than men, and the average age of voters is closer to her (even considering that older voters are more likely to vote). Plus, likely Clinton voters are closer to her demographically than likely McCain voters are to him. I don’t really see much of a difference with them on religion, and they’re both white. So I’d have take another tough bet and say Clinton would beat McCain based on demographics.
Now, obviously some groups of voters might vote more often than others, which I don’t necessarily account for. I’m also not considering a lot of ideological and political factors, which may sway the eventual outcome some. I’m just going with my oversimplistic one-factor model here, for better or worse. Even worse, as of now (early March), Obama’s contract to win the presidency is trading higher than McCain’s, which in turn is trading higher than Clinton’s. (48.5 to 37.1 to 18.4) Whether the Democrats nominate Obama or Clinton, my Presidential predictions are on losing bets (at least right now). Maybe I should’ve stuck to playing Monopoly.
| | | del.icio.us |
March 7th, 2008 at 7:19 am
Variations were said by several people:
‘It’s tough to make predictions, especially about the future” -Yogi Berra,
“… never make predictions – especially about the future” -Samuel Goldwyn.
“Prediction is very difficult, especially about the future” Niels Bohr
I figured it was Yogi Berra, but I wasn’t entirely sure. Thanks for the sources!
- Dave
March 7th, 2008 at 9:26 am
“I’m not quite sure who’s responsible for the quote that’s the title of this post, but whoever said it, I agree.”
That’s just one in a long line of quotes used to shame our current commander-in-chief, is it not?
“You go to war with the army you have,” sayeth Donald Rumsfeld. OK, not trying to get too political here.
- Dave
March 7th, 2008 at 9:30 am
“Curse you, Free Parking!”
Also, this makes me curious as to your “house rules” regarding Free Parking. I think the official rule is that it has no effect on gameplay - it’s intended simply as a “do nothing” space on the board.
We used to make it so it got all sales taxes, plus we put $200 on it or something. If you missed it when you needed it, it sucked.
- Dave
March 13th, 2008 at 2:22 am
Another stellar post. I am still baffled at how you manage to do this every week.
Thanks, Dan. Comments like these never get old and are always much obliged.
- Dave