Yep, I agree with this. I really built the model for fun, just to see how close it was possible to get with a few hours work.
A lot of forecasting problems have a structure like this one did:
- There is a lot of publically available data, and simple statistical models based on that data give pretty good predictions with very little effort.
- If you're an expert with a lot of time on your hands, you can put a lot of effort into improving the forecasts by 1-2%.
- In the end, it comes down to the effective business use of your model. Nate Silver had an excellent brand around his model.
In fact, I'd wager that he knows his model is more complex than it needs to be, but part of his brand is "rogue genius with a complex model that's far too difficult for mere mortals to understand".
A slightly less cynical take on the last point is that it also lets him rebut many objections with, "nope, the model already takes that into account". By having a piece of the model devoted to most possible talking points: convention bounces, whether undecideds break for/against the incumbent, effect of the economy, etc., he can claim he's addressed those critics' points, even if the net effect of addressing them is close to nil.
And an addendum, that 1%-2% accuracy is crucial in close elections like this one, so it's more than worth putting the extra time and effort into getting.
While your project got the right result, I don't think it's mathematically justifiable to take an arithmetic mean of the published survey means without accounting for the standard deviation of each sample distribution.
Doing that would make your model more mathematically complicated, but it's no doubt one of the things that Nate Silver is doing.
But besides that, Silver isn't just trying to call the election the Monday before the election, he's trying to build a model that predicts the outcome as early as possible.
A lot of the stuff he's been throwing into it, like the 'economic factors' or 'convention bounce' adjustments are about keeping the model from going off into the weeds due to predictable fluctuations in public sentiment caused by ephemeral events.
If you compare the history on the right column of the site to the "now cast" tab, which excludes those adjustments, you'll notice that the former contains fewer and more modest fluctuations than the "now cast".
You're right, it's not mathematically justifiable at all. The focus was on quick and dirty results over mathematical sophistication. Note that I described the adjustment for the number of polls as "a fudge"!
> Silver isn't just trying to call the election the Monday before the election, he's trying to build a model that predicts the outcome as early as possible.
That's true, but all the praise he's now getting is for correctly predicting the outcomes in every state - a prediction that he made less than 24 hours before the election took place (in fact, he changed his prediction for Florida from REP to DEM after I'd made my predictions, using up-to-the-minute polling data).
> In fact, I'd wager that he knows his model is more complex than it needs to be, but part of his brand is "rogue genius with a complex model that's far too difficult for mere mortals to understand".
Not what he said on The Colbert Report. "It's not hard math. You take averages then you count to 270."
part of his brand is "rogue genius with a complex model that's far too difficult for mere mortals to understand".
On the contrary, I think his brand is 'anyone who takes some time to study statistics & probability can work this out for themselves.' His key skill is demystifying the modeling and explaining that it's not the result of some secret sauce.
A lot of forecasting problems have a structure like this one did:
- There is a lot of publically available data, and simple statistical models based on that data give pretty good predictions with very little effort.
- If you're an expert with a lot of time on your hands, you can put a lot of effort into improving the forecasts by 1-2%.
- In the end, it comes down to the effective business use of your model. Nate Silver had an excellent brand around his model.
In fact, I'd wager that he knows his model is more complex than it needs to be, but part of his brand is "rogue genius with a complex model that's far too difficult for mere mortals to understand".