> The best criticism I can come up with is that 538 probably doesn't do a sufficient job of explaining to its audience that their published probabilities are predictions for what would happen if the event occurred today
I don't think that's true. While 538 does such predictions (the “nowcast”), they also predict actual results (the “polls-plus forecast” and the “polls-only forecast”), the default view is a forecast (I think the polls-plus usually, but I vaguely recall a chance to polls-only as the default during the 2016 cycle with concerns that there were reasons to suspect that some of the other signals on the polls-plus model had become less useful.) The “nowcast” is not the the default/headline number, so it would be very odd for a someone to mistakenly fixate on it thinking it was the forward-looking prediction.
This is not correct, or at least it isn't any longer. The current predictions by 538 are the "lite" (polls only), the "classic" (polls + other data driven indicators) and the "deluxe" (polls + other data driven indicators + expert opinion). All of these are predictions based on "what would be the outcome of the election be were it to be held today". They just incorporate different data and complexity in their models.
Previously, the reported predictions were the "nowcast", "polls-only", and "polls-plus". But despite their names, I believe those too, whatever 538 may have described them as, were "predictions for outcomes as if the event occurred today". In fact, were they not the case, then Taleb's argument would actually have merit, despite his obtuse attempts to explain it.
Essentially, his argument is that, if you are predicting a particular outcome to an event (say, Clinton winning the election) a year from now with a certainty of 90%, and 3 months from now, your prediction is 75%, and 3 months after that is 85%, and then right before the election, is 70%, is evidence of a flawed model (keep in mind, I'm making these numbers up).
If your prediction is 90% a year out, it should fluctuate much as time goes on, because 90% is very strong certainty. One may counter that as time goes on, public opinion may change, there are changes in news coverage, and external events like the economy or scandal can make a big impact, and all of that is absolutely true. But the point is that, if in light of new information coming forward, your model predictions repeatedly exhibit volatility, than your model failed to adequately account for the uncertainty as a result of future events, and you were never actually 90% sure in the first place. By necessity, a model predicting a distant event in which lots of unknown factors can cause it to fluctuate before it occurs will either be fairly stable as time goes on (meaning you indeed were correct that there was a high chance of a Clinton win), or should give much more hedged estimates of a win (meaning your probabilities a year out would be much closer to 50-50).
Taleb's point is that 538's models are too volatile to actually be accurate when they say there is an X% chance someone wins an election a year, or 6 months from now. However, his argument does not apply in the case where the prediction is that there is an X% chance someone wins an election, were the election held today. Instead he seems to dismiss that that type of prediction is worthless, which is simply wrong and foolish in my opinion. There is lots to be learned about the current political climate from those predictions.
So its entirely possible both parties are partially wrong and right. Taleb is correct that 538's models are too volatile to be accurate predictions far out from the election, but it assumes that the model is a prediction of what will happen on election day, rather than what would happen today. He also is throwing out the baby with the bathwater by insisting that that is the only kind of political prediction that has any value. On the other hand, 538 has not been very clear on if their forecasts are predictions of what happens on election day or what happens were election day today. If its the former, then their models have issues and Taleb has at least some validity to his argument. If its the latter, then Taleb's argument is no longer applicable.
I don't think that's true. While 538 does such predictions (the “nowcast”), they also predict actual results (the “polls-plus forecast” and the “polls-only forecast”), the default view is a forecast (I think the polls-plus usually, but I vaguely recall a chance to polls-only as the default during the 2016 cycle with concerns that there were reasons to suspect that some of the other signals on the polls-plus model had become less useful.) The “nowcast” is not the the default/headline number, so it would be very odd for a someone to mistakenly fixate on it thinking it was the forward-looking prediction.