Why Is Nate Silver So Afraid of Sam Wang?

Why is Nate Silver so scared of Sam Wang? Silver, who is legendary for his election forecasts, is the darling of political empiricists, sitting atop his personal empire of data-driven journalism at ESPN. Wang is a Princeton professor who also predicts elections, but he’s hardly a household name. So why won’t Silver leave him alone?

The two crystal ball gazers have been engaged in a running battle on Twitter, on their own websites, and in the media at large. Silver’s forecasts say Republicans will take control of the Senate in November; Wang’s have the Democrats maintaining their grip. But it’s okay for two guys to have different forecasts, right?

It isn’t for Silver. He’s been attacking Wang relentlessly, calling his methodology “wrong” and Wang himself “deceptive.” Silver could simply wait for the election results to come in and compare his forecasts’ accuracy with Wang’s across all the Senate races. Instead, he’s doing everything possible to discredit Wang before Election Day.

Here’s my guess at the reasons why. First, Silver fears Wang. In 2012, Wang’s model did a better job predicting the presidential election. Wang called not only Obama’s electoral college total of 332 votes, which Silver matched, but he also nailed the popular vote almost perfectly. Wang’s model also picked the winner in every single Senate race in 2012. It’s not good for business if Silver keeps coming up second-best.

But more importantly, Wang is the only one predicting Democrats will win. This represents a huge risk for Silver. If every forecaster had Republicans taking the Senate, then they’d all be either right or wrong in November; no one would have a better headline the next morning than Silver. There might be differences in the accuracy of predictions for each seat, but there’d be little embarrassment for Silver even if someone else happened to hit closer to the mark in a few races.

Yet with Wang in the picture, that’s not the case. If the Democrats hold the Senate, then Wang will stand alone; Silver will just be another one of the many who got it wrong. As of this writing, Silver’s own forecast says there’s a 41 percent chance this will happen. Imagine that -- a 41 percent chance that the whole empire comes crashing down.

This is why Silver hasn't spent much time dissing The Washington Post. Last week, the newspaper gave the Republicans a 77percent chance of winning; for Silver it was 58 percent, and for Wang it was 42 percent. That’s right -- the gap between Silver’s forecast and the Post’s was even wider than his gap with Wang. The big difference was that the Post posed no threat to Silver if Republicans won; he would have been right as well.

Silver clearly can’t tolerate this risk, and I think that’s why he’s spending so much time ridiculing Wang. If Wang’s forecast turns out to be correct, Silver needs the world to believe that it was luck.

Of course, a single election shouldn’t be grounds for validating or dismissing any statistical model. A 41 percent chance of an event is still a 41 percent chance; the event is quite likely to happen. No forecast is “wrong” unless it predicts something with 100 percent certainty that doesn’t end up happening. The way to evaluate forecasts and forecasters is by their performance over time -- how often do they miss the mark, and how often do they hit.

On this basis, Silver has little to worry about. He has a little egg on his face from making Brazil the overwhelming favorite in this summer’s World Cup, but overall his predictions were far more accurate than the bookmakers’ odds. And his history of forecasting elections places him among the elite prognosticators of all time.

But he probably knows that the public -- and the people who pay his salary -- might not be so generous. Like it or not, forecasting is a “what have you done for me lately” profession. It doesn’t matter that a 59-41 gamble is extremely tough to call correctly. After all, there’s a 50 percent chance that tossing 33 coins will come up with more heads than tails; 59-41 isn’t that far off.

So the battle rages on. Like Silver, I question the seemingly arbitrary cutoffs and weighting of the data Wang uses in his model. But I also wonder if Silver’s measures of momentum can ever forecast a turn in the polls. Over the years, I’m pretty sure both forecasters have benefited from luck, which is impossible to measure. We’ll see who’s luckier soon enough.