As I understand it, the feud in its current form started from a tweet from Dinesh D'Souza [1] in which he interpreted Nate Silver's statement that "Dems, GOP winning House are 'both extremely possible'" to mean that they had gone from an 80% chance of winning to a 50% chance.
Taleb inexplicably chose this as the hill to die on [2] and wrote a lengthy and technical paper to refute, and replied (on the [2] thread) "When someone says are event and its opposite are extremely possible I infer either 1) 50/50 or 2) the predictor is shamelessly hedging, in other words, BS."
I've often found Taleb's ideas to be profound and interesting, and unfortunately offset by his arrogance and occasionally cryptic pronouncements and colorful colloquial "Brooklyn-ese" intentional misspellings of words (like "gabish" for "capisce"). But in this case I'm having trouble following what his criticism even is -- forecasts, especially probability forecasts, are hard, and evaluating them is hard -- the dimensions of skill vs. calibration vs. precision are hard to evaluate, and Nate Silver does a fairly good job of describing the tension.
Yeah, "extremely possible" is perhaps a poor choice of words, and could be interpreted uncharitably as a hedge against any outcome, but I think there's an obvious charitable interpretation too: that things that have a 20% chance of occurring turn out to occur around 20% of the time. No one would be blown away by surprise if I proclaimed "SIX!" then rolled a six-sided die and got a six, but that's even less likely than 20%.
So it's reasonable to say that, if 538 claims that an event has a 20% probability of happening, and that event happens, no one should be astonished. Of course, such events ought to happen about 20% of the time, if 538's models are good. And that evaluation is precisely what 538 published a few days ago: https://projects.fivethirtyeight.com/checking-our-work/
Loads of people criticized Silver for not having "predicted" Trump's electoral victory. I imagine Silver's language was trying to hand-hold people to an understanding that even 20% probability events happen 1 out of 5 times.
With so many people trying to twist and spin every headline, prediction, number, I appreciate Silver's attempt to make a reasonable guess with transparent methodologies.
I have a hard time understanding this: what does it mean to assign probabilities to one-off events?? The only way to verify probabilities predictions is to test it a great number of times. Otherwise you can say anything and never be wrong (except for 0 or 100%).
Those questions underpin big chunks of philosophy.
Assigning a probability to a one-off event is enumerating all the ways it could happen, all the ways it could not happen and assigning a probability to each of those ways. Obviously there is a lot of guesswork; but if you need to make a decision based on the future that approach gives you a much better chance of making a good decision. In practice an event will be made up of components that are more predictable than the whole, and some real uncertainties. There is a lot to be gained by thinking hard about the situation, and assigning a probability will do that.
Silly example - how do I estimate my risk of falling climbing up a set of stairs that I've never climbed before?
* Baseline risk of tripping - I have a lifetime of data.
* Increase of risk being on a staircase - I have an area I want to put my foot on (a specific step) that is about 1/3 of the area that my foot usually falls in, so that increases the risk by an amount that can be reasonably estimated.
* I will watch my foot - maybe an order of magnitude improvement in precision.
This is enough to let me estimate the risk of carrying a bulky object (that obscures my view of my feet) up a staircase. I've isolated the uncertainty (how much does visual observation change the odds) from the certainties (areas, background rate).
Now I can take that to several experts who will identify new mechanisms and tighten up my estimations on how big a deal the components are. In this way - even though the final % I come to would still be a bit arbitrary - it is starting to become a summary of what a large number of people think about the inputs to the problem and their relative magnitudes. Being able to communicate all that thinking with a single number is a miracle in its own way.
It’s not that the election itself is subject to random uncertainty so much as the data that they are using to make the prediction is.
The data might favour one candidate, but even assuming it is unbiased and representative, it is only a random sample, and there is a chance it could be randomly wrong.
You can bet money on one off events. People and bookies do this frequently and you can see if they loose or win.
But the betting odds depend on how much knowledge you have - for someone with perfect knowledge the odds may be 0 or 1, if you know nothing you might guess 50:50 for practical purposes.
Every event is a one-off event. In particular the Trump election was the most one-off event in recent memory :)
Kidding aside, your observation is correct, you need to perform repeated predictions. 538 does just that. They keep predicting a whole lot of outcomes, so you can check their track record. They even have a challenge for the audience, where you can record your own predictions and compare them with their corresponding predictions (see for example [1] for NFL games). The scoring of the predictions uses the Brier score [2], which is just a version of R-squared for classification problems.
In the case of the November 2016 election, other prediction sites were giving the Trump team a 1-2% chance, 538 was giving them a 20% change. Considering that, 538 comes out as the clear winner.
Which brings us to what I consider to be the best measure of prediction accuracy (my own invention, it doesn't have a scientific name yet): you can compare two sequences of predictions (let's say yours agains 538) by simulating bets at some mid-odds level. For example if I say Trump's probability of winning 2020 is 45% and you say it's 35%, then I'm willing to give you 2-to-3 odds, while you're willing to take that. We then see who stays ahead most of the time in this betting simulation.
For what is worth, here's a recent survey of different scoring types for predictions: "Assessing the performance of prediction models: a framework for traditional and novel measures" [3]
> In the case of the November 2016 election, other prediction sites were giving the Trump team a 1-2% chance, 538 was giving them a 20% change. Considering that, 538 comes out as the clear winner.
One common mistake I've seen is people mistaking that 20% as the expected vote ratio. It definitely wasn't; that was hidden behind an apparently too-well-hidden toggle (buttons above the graph), and was floating around 45% Trump and 48% Clinton, with overlapping error ranges.
You can't verify probabilities for any single given event, but you can look at the outcomes for all events for which predicted probabilities were given, and use that to estimate the accuracy of the methodology/models used.
So, for example, when 538 said that Trump had a 30% chance of winning, its hard to say how accurate that was given it was a one time event. But if you look over all of their election predictions, and look at those they predicted to win 30% of the time, about 30% of them should have been victories (with some margin for error, of course). If 50% were victories, or only 10%, then maybe that would indicate their models aren't doing so hot.
The same people were criticizing him a day before because everybody else had Hillary at a 99% chance to win. Nate tweets that Trump is one standard polling error away from winning and people scoffed.
I remember him saying before the election that the odds of Trump winning we're roughly the odds of a randomly picked day being a Saturday. Which always seemed pretty clear to me - Saturdays definitely happen, Trump could definitely happen. And then it did, and it was nothing like a Saturday, not at all.
It is worth noting that 538 doesn't keep a historical record of their predictions, making it difficult to really see how they predict outcomes coming up to a conclusion. You can only see the latest/final prediction.
538 also predicted lower chances of victory in the weeks and months leading up to that, then adjusted it on election day vanishing away their earlier guesses. This isn't wrong on the face of it, but people infer from the final datapoint to the conclusion later that same day about 538's accuracy.
What I am saying is: 538 needs to keep a historical record of their predictions.
> It is worth noting that 538 doesn't keep a historical record of their predictions, making it difficult to really see how they predict outcomes coming up to a conclusion. You can only see the latest/final prediction.
It's worth noting that you are completely wrong: not only do they keep such a record, they also provide a handy chart of it on the latest prediction page so you can easily see exactly how the prediction changed over time.
The link I provided does actually show every single forecast 538 has ever made. OP was wrong when they said it was the raw data they use to make the predictions. Check out the link I provided, it has everything OP was asking for.
Please look at the actual data on the linked page. Each of the csv files on that page has thousands of predictions 538 made going back as far as 2008. This isn't the raw data they used to make the predictions, these are the actual predications that they made.
Edit:
For example, here are the headers on the file named presidential_elections.csv
Maybe this was true of past elections, but their 2018 house forecast had a “how this forecast has changed” section that showed their historical predictions throughout election season. It’s still visible here: https://projects.fivethirtyeight.com/2018-midterm-election-f...
> Maybe this was true of past elections, but their 2018 house forecast had a “how this forecast has changed” section that showed their historical predictions throughout election season.
It has not changed. They did this the whole way back when it was an independent blog before the NY Times (later ESPN then ABC News) picked them up.
That's not specifically a Brooklyn thing, it supposedly has more to do with the fact that most Italian families in the tri-state area came to the U.S. before Italy was a country with a standardized language. That's why e.g. in Connecticut they pronounce pizza as abeetz. For whatever reason knowing how to pronounce and spell Italian words "incorrectly" has become a point of local pride, even for people who aren't at all ethnically Italian.
For sure the pronunciation thing is real, either due to regional accents or linguistic divergence; I've heard it firsthand used unironically by Brooklyn and Philly Italian-Americans, as well as ample examples from television and movies. But writing it that way I haven't seen outside of Taleb (and Google seems to bear me out to some extent), and using it that way isn't wrong, per se, it's just an eye-rolling moment; someone trying too hard to show that it isn't book-learning, it's "street-learning".
> But in this case I'm having trouble following what his criticism even is
A piece by Isaac Faber [1] from December last year clearly described what Taleb's position is. A summary paragraph towards the end:
> Here we can say, with some confidence that FiveThirtyEight predictions are not always reliable probabilities. However, they masquerade as one, being between 0 and 1 and all. This is Taleb’s primary argument; FiveThirtyEight’s predictions do not behave like probabilities that incorporate all uncertainty and should not be passed off as them.
Faber's piece was discussed on HN [2] soon after it appeared.
I confess, I still don't follow the critique. Let's take the output of a skin-in-the-game model, like a binary futures market. This also predicts numbers between 0 and 1, that look a lot like probabilities, but by the argument presented here, they are not probabilities, and if you look at the output of a futures market, then you are being scammed because the market itself is not giving a decision boundary; it's just outputting the last trade price.
Yes, this allows a forecaster to say that even if the <50% candidate wins, that they weren't wrong, but so what? The forecasts should be reasonably well-calibrated -- that is, of all 20% predictions, 20% of them go in the 20% direction. 538 recently published a report of their calibration, which bears out their results (and kicked off the latest round of name-calling between Taleb and Silver).
Is the criticism just that we shouldn't look at Silver as a person with anything interesting to say because he won't commit 100% to one side of an argument? I honestly don't understand what the critique is, and this link I don't find helpful.
>I've often found Taleb's ideas to be profound and interesting
Really? Because financial crashes happen a lot more regularly than black swans. This seems like par for the course for Taleb; I've never read him in depth but he always seemed like a memer.
Taleb is simultaneously both worth reading as well as one of the most pretentious, unpleasant asses I've ever read.
The Black Swan was mostly fantastic, but Antifragile was mostly regurgitation.
The biggest problem with him is he thinks he's much, much smarter than he is, like a kind of inverse-Dunning-Kruger effect. It's odd because he is quite a smart man in the first place; there's no need for him to be such a pretentious asshole.
I've never read Black Swan. I found Antifragile hugely thought-provoking. Knowing both, would you imagine I had anything to gain from also reading Black Swan, or do you really think they basically cover the same ground?
There is a lot of overlap in the Venn Diagram between the two. I would say read the first 80-100 pages of Black Swan and decide if you would like to complete the book.
"Since Silver’s forecasts begin with probability models, it’s safe to assume they obey all the rules, including Bayes’, and would be arbitrage-free."
I'll do the author the courtesy of assuming they've got a more expanded version of their argument in their head and just failed to express that, but I don't think that follows at all. I don't think there's a constraint on "Nate Silver has a probability model" that mathematically guarantees "Nate Silver's probability model" is correct, in any sense of that term. One could certainly provide evidence and/or proofs that Nate Silver's model follows those rules, but I don't think it just goes without saying. If nothing else the model could contain an straight-up error; goodness knows that's easy enough.
(If one wishes to add to the concept "probability model" that it must be correct by definition, than I can't accept the statement "Nate Silver has a probability model" with mathematical levels of confidence. He claims to have one, yes, but that's a long way from a proof.)
> I don't think there's a constraint on "Nate Silver has a probability model" that mathematically guarantees "Nate Silver's probability model" is correct
If I'm understanding the article correctly, it's not talking about models being "correct," it's just talking about the probabilities of all the outcomes adding up to 100%, which means you can't place a bet that guarantees you make money regardless of the outcome.
In other words, offering a bet that was based on the claim that there is a 99% chance of rain today in the Atacama Desert, and a 1% chance of no rain there, is arbitrage-free. It's still clearly a poor probability model for one of the driest places on Earth, but it's arbitrage-free.
If over time, your forecasts predictably change, then you can be arbitraged.
If e.g. I know that you'll have a probability below 40% at some point in the future, and I know you'll have a number above 41% at some point in the future, then it's trivial to create a strategy over time that is guaranteed to make money.
Taleb's claim, as I understand it, is that Silver is incorporating random noise into his model and so will exhibit such predictably swings, which enables arbitrage.
If over time, your forecasts predictably change, then they are not the best forecasts you could make with the information on hand (I take it that by "predictably change", you mean that they will change in a predictable way, not just that it is predictable that they will change somehow.)
If I am not mistaken, an opportunity for arbitrage is not created merely by a certainty that they will change somehow; it would require that they change in a specific way, and deterministically so. Absent this, a bet would just be speculation, not arbitrage.
>If over time, your forecasts predictably change, then they are not the best forecasts you could make with the information on hand.
And this is precisely the complaint Taleb is making.
An arbitrage is just a strategy that's guaranteed to make money. If you know for sure that something will go up and down at different points you could make money risk free. Simply buy at any point below the current price and sell at any point above.
> An arbitrage is just a strategy that's guaranteed to make money. If you know for sure that something will go up and down at different points you could make money risk free. Simply buy at any point below the current price and sell at any point above.
This is not arbitrage. Also this is precisely the kind of strategy Taleb would have argued against in his book about black swan events.
If someone was flipping a coin and raising their price when it landed heads and lowering it when tails, arbitraging using reversion to the means would be a perfectly legitimate arb strategy.
Taleb's argument is that Silver's estimates are based on random noise, and are thus vulnerable to this form of arbitrage.
It think it’s… sort of the reverse? Flipping a coin results in Brownian motion, which is an example of a “martingale”, a function whose expected future value is equal to its current value; and the evolution of a share price being a martingale (at least under a modified probability measure) is exactly the condition for the market to not have arbitrage:
I’m not sure exactly how the arbitrage works, though.
But the argument in the paper seems to be not that there’s noise but that there’s too much of it. In fact, it assumes that the expected vote share over time is subject to Brownian motion with specific parameters (which is a rather flawed assumption), but then concludes that 538’s probability distribution is more swingy what he’d calculate from those parameters.
The point is that Silver's predictions are the result of coin flips, but that the underlying probability isn't. If he's updating further than he should then you can make money by updating the correct amount and betting against him.
A price set by coin-flips does not revert to any mean. It's a random walk and it diverges. The idea that heads and tails must eventually even out is a common fallacy.
Oh so the coin-flipper is not setting the price, but trying to predict it? I think at least I understand what you're explaining. I don't see how that is a useful model for what is going on though.
Predicting the result of a vote for a day when it's not going to be held is a form of entertainment that can't be verified. Is Silver claiming any more than that?
Yes, Silver is absolutely trying to forecast elections months in advance. Taleb's argument is that the polls at that point are largely noise, and so Silver is updating too much on them when he should basically ignore most of them until shortly before the election.
Silver's forecasts do predictably change, more precisely in that they swing deterministically in favour of one candidate whenever the most recent poll to come in favours that candidate heavily.
Each time the probability drops below 40% (in your example) it is likely because the fair odds swung in the latest polls. To attempt to take advantage of this is to ignore the recent evidence (conditional upon that evidence being quite different to the current concensus).
It's not at all unreasonable to do so, but isn't arbitrage. What if all polls started being increasingly conclusive leading up to election? 40, 39, ..., 0. No surprises and no arbitrage.
Perhaps another way to express this critique of Silver's methodology is that it lends too much weight on recent outlier observations? Sure if you knew this you could profit off these swings, but making the arbitrage argument doesn't add much to the discussion to my mind.
So what you're saying, I guess, is that Silver, by dint of making point probability predictions, is offering a two-sided market with infinite liquidity trading at that value for a binary option, and you could make arbitrage gains by trading off of volatility?
I'm having a fair bit of trouble picking up that from either Taleb's or Clayton's article, but let's say that it's a fair assessment.
Is the complaint that the observed variance is high enough to enable these arbitrages? So in order to make his model arbitrage-free, Silver would have to provide a spread rather than a point price, with his forecast as the midpoint of the spread, and the width of the spread wide enough that .5 is always inside the spread, thus making the forecast economically useless?
Or is it that Silver does not provide error bars that would estimate that uncertainty?
Or is it that because of intrinsic uncertainty in the election?
Well obviously Silver isn't actually offering to bet at those odds.
But complaining that a model is incorrect because it enables arbitrage is a standard complaint, it's how you can prove all non-Bayesian models are irrational. That doesn't mean all Bayesian ones automatically pass the test though.
Error bars and spreads are already outside the Bayesian paradigm, which demands a single number.
Ah, interesting link. I think I ran into some confusion since 'rational' and 'Bayesian' are somewhat overloaded terms.
When I think of Bayesian analysis, I always think of Bayesian models which attempt to model uncertainty and always have a posterior distribution (spread) of outcomes.
For example, in the Dutch Book page, instead of having a point-prior (P=0.51 (delta on P=0.51?)), each degree of belief could have some (different) distribution which could yield a posterior with a distribution and change the math here.
I think most bayesians would disagree with you on this point :)
It's not incoherent to analyze the posterior distribution by reporting a credibility interval instead of a single number (for example the posterior median).
An interval to estimate a parameter is different from an interval to represent a probability. You can't have a range for a probability, you can have a range for a parameter you're trying to estimate.
Well, in theory you can divide the uncertainty into two parts, by supposing that the outcome is driven by a fundamentally stochastic process and you also have imperfect information about the parameters of that process. For example, the outcome could be determined by a coin flip but you’re not sure whether the coin is fair. In that case, there is a “true” probability of heads based on the nature of the coin, and separately, a Bayesian observer can have a probability distribution for (i.e. representing their beliefs about) the value of the true probability. The observer could then come up with a single number representing their belief in heads by taking the expected value of that probability distribution, and if they just want to gamble on the outcome, that number would be all they need. But in order to correctly update their beliefs given future information about the parameters, they have to remember the original probability distribution; they also might just be curious about the nature of the underlying stochastic process, in addition to the final outcome.
How well that models an actual election is debatable, of course, but I think it does model it to some degree. In reality, there are not two stages but multiple, and none of those stages are necessarily fundamentally stochastic; rather, you just need exponentially more information to predict one stage than to predict the previous one, and without that information you may as well treat it as stochastic. For example, if I’m about to flip a coin, a god with exact knowledge of the state of my body and brain, the air currents in the room, etc. might be able to predict how I’ll throw it and how it will fall, but mere mortals have to treat a coin flip as random. Similarly, a god with exact knowledge about the state of the universe might be able to predict an election result eons in advance… though quantum randomness might trip them up. Getting more down to earth, if you just could poll every American about their political beliefs, you could make much better predictions than you can with real polls, which have to take random samples and thus accept some level of stochastic polling error. On the other hand, polls can also suffer from methodological error, which is fundamentally different in nature; it can be highly pernicious, but does require a smaller quantity of information to correct for. And so on.
Sure, but as a good Bayesian you should be willing to bet based on that expected value, not based on the spread.
Re polls: still more complicated than that, even with 100% polling you'd still have response error and non-voters. Much of the difference between polls is their assumptions about demographics of voters.
Sure, but that doesn’t mean 538 can’t include information about the spread, in addition to their top-line number which is just a single probability.
And yes, my discussion of polling error was oversimplified, but I was mainly just trying to explain what I meant about having to treat something as random if it’s theoretically predictable given enough information.
EDIT: Hold on; I don’t think 538 does include error bars for their probabilities. They have error bars for measurables like the vote share, but that’s normal. So I don’t understand what this argument is even about :)
Someone several comments up suggested Silver should include error bars and should bet based on a spread, I said that was not proper Bayesian methods, then we had this back and forth.
I thought that bit was very well-expressed. Let’s see if I can explain what I think I understood by it...
If you’re buying and selling bets on a range of events, you can only be immune from arbitrage if all your bets are mutually consistent. For example, if you’re offering 3:1 odds on event X, you can’t offer more than 1:3 on the complementary event ~X (where exactly one of X and ~X must happen). That’s pretty obvious, but there a lot of much trickier variations.
“Nate Silver has a probability model” just means his forecasts are derived from a self-consistent model about the probabilities of all the micro-events that make up the election. That fits with the information FiveThirtyEight have published about their methods, so it seems correct to me.
One way Silver’s model could be invalid is if there’s just a bug, and a single event is tracked in multiple inconsistent ways. For example, if the model unknowingly projected odds for the same politician in both the presidential election and a senate seat, without accounting for the fact that you can’t win both at once. But if that’s the criticism Taleb is making, he should be able to put his finger on a specific bug, which he doesn’t seem to have done.
Another way Silver could be wrong is if his models are far too volatile, so you know if the probabilities shoot up or down a long way they’ll almost certainly revert to the mean again. I imagine you could run arbitrage against that.
It’s far from clear that that’s the case, though. It could be that his models are volatile but still pretty close to reality, in which case just betting against every big swing in Silver’s model would be like trying to make money by predicting day-today movement on the stock market.
Isn't Taleb arguing that his model comes being modeled during periods of relative calm. So when the black swan event does happen, that wasn't priced in as a possibility at all, creating an arbitrage there.
That sounds like it might be right. I think another commenter here correctly identified that it’s a misunderstanding -- Silver’s model estimates “the election result of the election were to happen tomorrow”, so unlikely events in the coming months don’t need to be accounted for by definition.
If so, I think there’s a valid criticism that the “now-cast” notion is a bit deceptive, and also impossible to validate (we’ll never know what that hypothetical election result would actually have been).
So... I wrote this. Maybe I can clarify a little what I meant by the statement "Since Silver’s forecasts begin with probability models, it’s safe to assume they obey all the rules, including Bayes’, and would be arbitrage-free.", since this seems to be confusing some people:
What I'm addressing here is Taleb's claim that Silver's probabilities would allow for arbitrage (i.e., riskless profit, not just profit on average) if turned into betting prices. This is in the sense of arbitrage-through-time, buying low and then selling high. As I discussed in the piece, an old argument due to de Finetti says that if prices are arbitrage-free they must satisfy the equations of probability, meaning you can in some sense think of the price as giving a probability of the outcome. For time-dynamic arbitrage the relevant equation is Bayes' Theorem. All I meant by my statement above is that the converse to de Finetti's argument is also true, trivially. If prices obey the equations of probability they are automatically arbitrage-free. And since Silver's probabilities begin life as probabilities, they satisfy all the relevant equations (one would expect).
Technically, the way we'd express this in modern finance is through the Fundamental Theorem of Asset Pricing, which says a (complete) market is arbitrage-free if and only if there exists an equivalent measure under which asset prices are martingales. Silver's probabilities are necessarily martingales, just because of the way conditional probability math works, so unless Taleb can claim that his and Silver's probabilities aren't equivalent, meaning they disagree on what events have probability zero, then there is no chance of arbitrage. That's just the mathy way of saying the same thing I said in the post.
There are many other possible errors Silver could be making, and many other possible criticisms Taleb could have made but did not. In this case he wrote a paper claiming a mathematical result that just isn't true.
These are consistent (or behave as probabilities or what have you) at each time (separately), so de Finetti's argument covers each (separately). Does this alone somehow protect from arbitrage over time? Or is the martingale stuff in the next paragraph, though postured as only a technical rephrasing, essential? Unsure since that part's over my head.
The above's enough to pose the question, but continuing for concreteness: if a buyer can determine any "significant" pattern in my assessments over time--for instance maybe my past assessments have been routinely seen to tend to a uniform distribution over time--aren't I still vulnerable to arbitrage, or else what's protecting me?
So, if I knew ahead of time what your price was going to be tomorrow, I would have a clear arbitrage strategy. But that's not a fair example. The question is: what do I know at t=0 about what your price could be at t=1?
A better example is this: suppose you're creating a probabilistic forecast of the chance of a coin coming up Heads twice. Suppose this represents a win. You say, "according to my model, P[H1 and H2] = 0.3." And suppose you also say P[H1] = 0.4
Then I ask you: imagining the first flip comes up Heads, what will your price for the second Heads be then? The only arbitrage-free price you can quote me is your conditional probability: P[H2 | H1] = P[H1 and H2]/P[H1] = 0.75. Anything else allows me to buy and sell bets and make a riskless profit.
Now, note that if H1 occurs, P[H2 | H1] is now the updated chance of getting H1 and H2. You observed some information and updated your probability accordingly. Also note that according to your model, the chance of H1 happening was 0.4, so the possible prices of "H1 and H2" after one coin flip were 0.75 with probability 0.4 and 0 with probability 0.6 (if the first flip were tails, "H1 and H2" is impossible). So the expected price of "H1 and H2" after one flip is (.75)(.4) + (0)(.6) = 0.3, which is also your price at t=0. That means the probability/price is a martingale according to your probability assignments, which, according to the Fundamental Theorem of Asset Pricing, again means no arbitrage is possible, unless we disagree on what events have probability zero.
So, the election forecasts are like this except we don't get to see all the inner workings. Nate doesn't quote things like the probability of the polls moving by a certain amount each day. We just get to see things like P[H1 and H2] before and after the first coin flip. But as long as it's possible that those come from a consistent set of conditional probabilities and Bayesian updating, there's no chance for arbitrage.
Isn’t it just one big misunderstanding between them?
Taleb thinks 538’s probabilities represent a binary option price on the event, in which case, yes, the probabilities should stay very close to 50% because the vol is so high. Whereas Silver’s models are actually saying “Based on current polls, if the election were held tomorrow, then the probability candidate X wins is Y%” and are thus allowed to swing more wildly. Aren’t both just fundamentally different things or am I missing something in Taleb’s argument?
This makes perfect sense to me. But it shouldn't be 50%, it should be "regression to the mean" where the mean is a "naive" forecast based on history, the economy, demographics, etc, etc, etc.
Silver is making a prediction based on the best model that he has.
It isn't quite arbitrage, but if you did binary option pricing then you could indeed fairly safely make money off of Silver by taking known events that you know swing polls, like conventions, and betting on the likely pricing changes.
That said, the data set that Silver is developing using his model is going to be one of the key inputs into a more sophisticated pricing model. And how to factor in those other external factors is going to require a lot of calibration between Silver's predictions and a known database of such external factors. Which means that Silver's approach is the right starting place to get there, no matter how much Taleb might wish it otherwise.
A forecast that really did try to be an option pricing model like that would be interesting to see. It would have the advantage over a “now cast” that you could actually run the numbers and see how accurate it is. Whereas nobody can ever know what would actually happen if there were an election now rather than a year from now.
> A forecast that really did try to be an option pricing model like that would be interesting to see.
Silver’s election forecast models are exactly that. The nowcast expressly is not, but it's also not the headline model.
> It would have the advantage over a “now cast” that you could actually run the numbers and see how accurate it is.
That's true, and not just in theory: Silver has recently run the numbers for all of 538s forecasts (together and separated by sports vs. elections) and they are relatively accurate but not perfect; the supporting data is available for download, too.
Aha, thanks for clarifying that. I actually read that article, but the earlier comment got me confused over whether it was merely validating the final forecasts, or all forecasts over time.
Makes sense that it was written as an explicit rebuttal to Taleb -- but without mentioning him directly. :)
I don't think your assessment of 538's models is correct. They used to have something called the "nowcast" which is what your referring to, but they got rid of that.
As was mentioned elsewhere in this thread (and I am aware of from reading 538) during the campaign, Silver updates separately both a probability based on the scenario of "if the election were held tomorrow" and the best prediction of the election on its actual date.
In the 2016 election season, Silver used three models, two forecast models predicting what the election would do (a “polls-only” and a “polls-plus” model incorporating non-poll data), and the “nowcast” of what would happen if the election were held at the moment of the analysis. The polls-only forecast was (at least at the end of the cycle) the headline result.
Taleb calling someone else an entertainer is a bit rich, sort of like a clown telling someone to have dignity. I have no opinion on Silver whatsoever, but Taleb is a bloviating ass who has gone from popular milquetoast observations, to climbing fully up his own backpassage. That this all began with the likes of Dinesh D’Souza doesn’t help.
I realize that Black Swan is popular here, but it was horrendous. A single essential premise which, instead of support, rested for chapter after chapter on assertions. That’s not evidence, it wasn’t an argument, it was the sound of someone having one good idea and then realizing they lacked the capacity to support it.
Quantifying the current voter sentiment makes you a entertainer? If anyone should be accused of "entertainment", I'd imagine it's the guy known for writing mass-market books.
GP doesn't even have Silver’s model right (he describes the “nowcast”, which was never the headline forecast model), so if he is right that that is what Taleb is referring to, that would be a pretty damning indictment of Taleb’s critique.
Nassem Taleb is an arrogant asshole who can't write. I tried reading "The Black Swan" and gave up in a flurry of swearing after less than 100 pages because he utterly failed to justify any of his breathless assertions with anything resembling an argument. Assertion after assertion with no reasoning linking them. Game to Silver on points. At least he can back up his arguments with explanations.
I really want to be Nassim Taleb, I don't want to get a real job, I don't want to be a slave for any kind of rules or laws whether in corporates, government or academia, I am much bigger than that, I just want to make enough money trading and shorting stocks during an economic bubble and retire young and then set myself emperor of the world and make fun of whoever I don't like such those academics who can't make 6 figure until in their 50s, of course academics are much less smarter than a stock trader, otherwise why couldn't they become rich like Taleb? right?!
Well, you're being sarcastic, but he does have a point in all that.
The proof of the pudding being in the eating, and all that.
There's something to be said for respecting somebody who betted on something he called and won (especially if they did it repeatedly), especially when most others were all against the very possibility of them being right...
And being free with fuck you money does give you some freedom to say truths (even if they're just your personal truths) employees can't.
even though I am kinda sarcastic I extremely admire this kind of thinking (assuming that I got him right), many stock traders have this kind of free thinking mentality and disdain for all kinds of rules and authority, George Soros once said that the only reason people read about his philosophy is that he made a few billions, he knew from the very beginning that having enough money would make his ideas worth of being considered for smart people and absolutely right for retarded people. It's like money shifts the truth or something!
If I understand Taleb's argument, it can be boiled down to the following:
A year before an election, there's 365 days worth of news stories, interviews, scandals, and other stuff that will be released to the public before election day. The day before the election, 364 days of that election influencing stuff has already happened and the effects of it are now known. As you get closer to an election, you have more information, and there is less time for public opinion to change, so it follows that you can be more certain of the election's outcome.
Taleb's assumption that voter psychology can be modeled with a random walk is pretty questionable, but if you give him that assumption, what's wrong with his math? The article doesn't actually address this.
Each person's claims are either right or wrong (assuming they are objective claims), regardless of any attribute or action of the person making the claim.
Russ Roberts invited them both to discuss their differences on econtalk- Nassim promptly declined. Until that happens, I think the twitter feud is without value.
Since Taleb delights in make his mathematical writing as opaque as possible, it's a useful read in just to know exactly the claims that Taleb is making in his paper.
In regards to the very specific claim that the volatility of an option's price decreases asymptotically as the uncertainty of its underlying increases? No, that's entirely correct.
In regards to Nate Silver's forecasts more generally? I don't know - it's hard to get past the tone of his arguments to understand what his actual disagreement is.
He is wrong in that the sentence of his that I quoted contained two statements: "when the volatility of the underlying security increases, [1] arbitrage pressures push the corresponding binary option to trade closer to 50% and [2] become less variable over the remaining time to expiration." His calculation proves [1], but it's [2] that is the basis of his criticism of Silver. And it's just not true, as can be seen even in a simple random-walk model.
"[2] become less variable over the remaining time to expiration ... And it's just not true, as can be seen even in a simple random-walk model."
Dear @aubreyclayton, I'm genuinely interested in seeing how you arrived at this conclusion. Would you kindly share the proof, or at least explain the logic behind it. Thank you.
If that's foreign to you, you might be interested in this write-up I did about election-forecasting in which I considered the same example, just in discrete time rather than continuous time:
http://nautil.us/issue/70/variables/how-to-improve-political...
The basic idea is that if you increase the volatility of a random-walk process, say by making the step-sizes larger, that won't actually make the probability of finishing above where you started any less (or more) volatile. The higher volatility means that from any given starting point your final resting place is more dispersed, but you're also more likely to range farther from home as you go. The two effects exactly cancel. Taleb's critique misses the second part of that.
From a business perspective, the volatility of Silver's models seem like a feature - they enhance the drama / sensationalism of election coverage. One week, he's telling me candidate X will likely win, the next week it's candidate Y, and I'm on the edge of my seat.
If Silver really believed these probabilities were correct, he should be willing make bets with these odds, otherwise his incentives are distorted.
> If Silver really believed these probabilities were correct, he should be willing make bets with these odds, otherwise his incentives are distorted.
Yes, the "skin in the game" argument. That is Nassim Taleb's calling card. Personally, I consider it a little unreasonable to expect people to literally put their money where their mouth is every time they make a forecast.
It's a kind of ad hoc censitary suffrage. One can only put possessions on the line if they have them, and the greater one's possessions, the greater their perceived "skin".
Isn't that just new polling data being introduced into the model? It seems disingenuous to say Nate Silver is being sensationalist when he's applying fresh new data to his models and seeing changes in the outcomes.
Have they though, or have they only made you feel as though your knowledge of probability and statistics has expanded? Taleb is the poster boy for "sound and fury, signifying nothing"
I don't know about Taleb, but Nate Silver's book on Bayesian statistics and prediction in general got me really interested in statistics from a general-reader standpoint. Where I ended up digging harder into the more math/sciencey books afterwards.
The part about predicting Earthquakes and his work on sports is really good.
I agree about Silver, and that was kind of my point! Silver knows his shit and is able to communicate it in such a way that the reader actually learns something. Taleb may or may not know anything, but all he is able to convey to the reader is the impression that they've learned something (however, he is very good at giving the reader an unearned sense of intellectual superiority).
Sure but he sounds like a decent scientist to me. While it’s true his organization is very much an entertainment vector rather than a more academic science output, which lowers the bar, that doesn’t necessarily mean the work he’s doing doesn’t have some utility and value.
Probably a lot more real value than a typical news outlet these days which pump out garbage at high speeds.
Taleb expanded my view of the impact of fat tails. (And my interpretation of his book helped me in an interview) He also helped me realize the limitation of models.
I stopped paying attention when Silver said the Buckeyes only had a 2 percent chance to win the title, because I knew that was bogus, team was stacked. Then he did the same thing with the Cavs when they turned around and won, so it just seemed to fit what Taleb says about "predictors" in The Black Swan. Since then, I don't listen much to forecasters, their track record is dubious.
>The converse to the statement above is also true: if prices form a set of probabilities, then they are consistent. Since Silver’s forecasts begin with probability models, it’s safe to assume they obey all the rules, including Bayes’, and would be arbitrage-free.
I don't believe this is correct. You can lose money to arbitrage even with a calibrated model if you ignore data someone else has, or include random noise someone else isn't including, which is what Taleb's argument is.
Someone betting Taleb-style would have an edge over someone directly betting on Silver's forecast numbers, but technically that's not arbitrage -- arbitrage in the sense of the article means a strategy with guaranteed return, not just very probable. Is that right? So my impression is that this is an argument about words, where traders mean 'arbitrage' more broadly.
(1) Most of Nassim Taleb’s math is more branding and showing off than actual science or statistics. It has been stated by people who are far smarter than Taleb that “if you can’t state it simply then you don’t understand it.” If Richard Feynman can explain physics in a relatable way, then taleb can talk about statistics in the same. He doesn’t because explaining is not his objective, personal branding is.
Taleb uses obscure, nonsensical math to show off and brow beat opponents. In my experience, working with people who are actually smart, this is not what very educated and intelligent people do.
The general public may fall for pictures of pages of obscure nonsensical calculations depicting god knows what, all I see is fraudulent personal marketing.
I say this having worked for and known CEOs who use the same tricks to lie to investors. It’s bullshit.
(2) Nate Silver runs a media opinion network. I followed his work and early on they did very rigorous analysis and had great and accurate visualizations which did seem to accurate predict results.
For whatever reason, I saw five thirty eight evolve into a bunch of media talking points and opinion mongering. The coverage on five thirty eight for the 2016 election resembled strongly that of standard news networks.
Why? Because if you want to make money on advertising you need to completely fill the airwaves with as much content as possible and opinion content is the cheapest and easiest to produce.
My opinion is that Nate slipped. His data analysis got distracted by running a media business, which focuses on advertising revenue and being likable by target demographics.
Frankly I don’t trust either of them. I also side more strongly with Taleb - predicting outcomes with random hidden variables is an exercise in chicanery.
No one wants to hear that the universe is random and that predicting it is impossible for long stretches of time. Look at the comments here, 90% of people (probably programmers) think that if they can just find the right equations then predicting the future is possible.
I really like reading Nassim Taleb's books, but he appears to have all of the social skills of, say, Linus Torvalds, when interacting with others in a public way.
Well I know for a fact that Ann Coulter is on very good terms with several HuffPo dems, yet they appear bitter rivals on tv. Its definitely plausible Taleb, Silver & Pinker are drinking buddies who hatched up this elaborate charade to hawk stat books. Statistics is an extremely dry subject & nobody including the doctorate students are particularly excited about, so props to this triad for engaging in these antics.
Possible... but I don't think so. I believe the hatred is pretty visceral. For one Taleb is/was pro-Trump, Silver definitely not. Taleb, despite being a professor/author/elite in his own way is pretty anti-elitist in rhetoric, again very different from Silver. Personality-wise, Taleb is pretty abrasive and has very few 'friends' in the public sphere that I know of, despite the high profile of his work--this is regardless if you think he is right or wrong.
I am not so sure about this. When you read his books or look at his Twitter feed you surely get the impression he is a self-obsessed, extremely arrogant weirdo. But when he gives e.g. a talk in person, he comes across very differently - self-reflective, engaging, emphatic and funny. As someone else has mentioned in this thread, there are really two Talebs out there...
Postmodernists think that being upset about someone's personality changes the very fabric of spacetime. "If he's mean, he must be wrong." That's hardly the case.
I think nate silver is bad math because when he can capitalize on drama he does. He paints himself as a fortune teller via math but has clear demonstrable bias. When you do that as a predictor then you've thrown out your integrity.
I have zero respect for election forecasters because I think they are not any better than astrologers. Nate silver has used the word probability in a way that is not consistent with mathematical definition of probability so often in this context.
Taleb despite being brilliant mathematician is actually pretty mediocre when it comes to real world. A lot of his "discoveries" such as "skin in the game" are well known to economists in the form of various theories.
I once exchanged few tweets with him about his criticism of Pinker's claim that we are probably living in one of the most peaceful times in human history. At no point Pinker makes a claim it was inevitable or it being the evidence that peace will continue to exist. Taleb wrote a length paper on the topic which I found to be totally incoherent. When I asked questions his retort was that I do not understand statistics.
The best criticism I can come up with is that 538 probably doesn't do a sufficient job of explaining to its audience that their published probabilities are predictions for what would happen if the event occurred today.
This seems to be one of Taleb's (and others') criticisms of their predictions: that they don't consider the probability of major unexpected events occurring between now and the event (like an athlete getting injured in the future, or the FBI announcing an investigation of a political candidate). But as I understand it, 538 is explicitly not attempting to consider those things, and is simply publishing probabilities of outcomes if the event were to happen now.
> The best criticism I can come up with is that 538 probably doesn't do a sufficient job of explaining to its audience that their published probabilities are predictions for what would happen if the event occurred today
I don't think that's true. While 538 does such predictions (the “nowcast”), they also predict actual results (the “polls-plus forecast” and the “polls-only forecast”), the default view is a forecast (I think the polls-plus usually, but I vaguely recall a chance to polls-only as the default during the 2016 cycle with concerns that there were reasons to suspect that some of the other signals on the polls-plus model had become less useful.) The “nowcast” is not the the default/headline number, so it would be very odd for a someone to mistakenly fixate on it thinking it was the forward-looking prediction.
This is not correct, or at least it isn't any longer. The current predictions by 538 are the "lite" (polls only), the "classic" (polls + other data driven indicators) and the "deluxe" (polls + other data driven indicators + expert opinion). All of these are predictions based on "what would be the outcome of the election be were it to be held today". They just incorporate different data and complexity in their models.
Previously, the reported predictions were the "nowcast", "polls-only", and "polls-plus". But despite their names, I believe those too, whatever 538 may have described them as, were "predictions for outcomes as if the event occurred today". In fact, were they not the case, then Taleb's argument would actually have merit, despite his obtuse attempts to explain it.
Essentially, his argument is that, if you are predicting a particular outcome to an event (say, Clinton winning the election) a year from now with a certainty of 90%, and 3 months from now, your prediction is 75%, and 3 months after that is 85%, and then right before the election, is 70%, is evidence of a flawed model (keep in mind, I'm making these numbers up).
If your prediction is 90% a year out, it should fluctuate much as time goes on, because 90% is very strong certainty. One may counter that as time goes on, public opinion may change, there are changes in news coverage, and external events like the economy or scandal can make a big impact, and all of that is absolutely true. But the point is that, if in light of new information coming forward, your model predictions repeatedly exhibit volatility, than your model failed to adequately account for the uncertainty as a result of future events, and you were never actually 90% sure in the first place. By necessity, a model predicting a distant event in which lots of unknown factors can cause it to fluctuate before it occurs will either be fairly stable as time goes on (meaning you indeed were correct that there was a high chance of a Clinton win), or should give much more hedged estimates of a win (meaning your probabilities a year out would be much closer to 50-50).
Taleb's point is that 538's models are too volatile to actually be accurate when they say there is an X% chance someone wins an election a year, or 6 months from now. However, his argument does not apply in the case where the prediction is that there is an X% chance someone wins an election, were the election held today. Instead he seems to dismiss that that type of prediction is worthless, which is simply wrong and foolish in my opinion. There is lots to be learned about the current political climate from those predictions.
So its entirely possible both parties are partially wrong and right. Taleb is correct that 538's models are too volatile to be accurate predictions far out from the election, but it assumes that the model is a prediction of what will happen on election day, rather than what would happen today. He also is throwing out the baby with the bathwater by insisting that that is the only kind of political prediction that has any value. On the other hand, 538 has not been very clear on if their forecasts are predictions of what happens on election day or what happens were election day today. If its the former, then their models have issues and Taleb has at least some validity to his argument. If its the latter, then Taleb's argument is no longer applicable.
"p.s. You continue to cite a paper in which you critique our now-cast (a deliberately volatile hypothetical of what would happen in an election held today) and mistake it for our forecast but because you're such a stubborn idiot you won't admit to this very basic mistake."
That seems like a pretty big difference to gloss over. Why are they publishing probabilities for if the event were to happen today? The event isn't happening today!
There are two types of people consuming these projections. I'll call them the sportsbook type and the newspaper reader. The sportsbook consumers want to know what the probabilities are for the actual event, when it occurs (election day). The newspaper consumers want to know what is going on currently and the country's political temperature.
538's forecasts, especially far from election day, are for the newspaper consumer. They provide insight into current political dynamics, and a quantitative look at how people view policy, the economy, where current fundraising is at, etc. in a single concise number that can serve as the basis for further discussion and analysis. They however aren't useful for the sportsbook consumer, because they don't account for the practically innumerable "unknown unknowns" that will occur between now and election day (scandals, terrorist attacks, fluctuations in the economy, flubs on the campaign trail, external activist efforts gaining traction). If they did account for all that uncertainty, then the predictions consistently be near 50-50 for the majority of the race, which not only is still uninteresting for the sportsbook types, but also means you no longer are providing value to the newspaper types.
Its important to point out that, over time, the predictions become applicable to both readers. As election day nears, the number of "unknown unknowns" decreases, and a prediction for "what would happen if an election were held today" and "what will happen on election day" converges. So up until the few months and weeks leading up to election day, the predictions should be taken more as commentary, and then perceived as serious indications of what will occur closer to election day.
They don't exactly gloss over it. They have tons of articles explaining it, but of course the average reader is just going to see the probabilities and run with that, and 538 does have some responsibility to actively explain things upfront in their UI, which I don't think they do a good job at.
> Why are they publishing probabilities for if the event were to happen today? The event isn't happening today!
My guess is that it's extremely out of scope (and probably ultimately infeasible) to try to incorporate these things into their models. Sure, there's a non-zero chance of a candidate being killed by a meteorite, but I think it's fair to ignore that and, if it happens, admit that you didn't even attempt to include the probability of that into your model.
> the average reader is just going to see the probabilities and run with that
The putative “average reader” who does so will see the default view, which is a forecast, not the nowcast, which you must actively shift to in order to see.
> and 538 does have some responsibility to actively explain things upfront in their UI,
Like, on the view selector that you have to use to choose the nowcast instead of the default forecast, where the nowcast is described as “Now-cast / Who would win the election if it were held today”.
Your personal meteorologist presumably predicts rain on days in the future, when it could happen, not days in the past.
Nate publishes predictions for events that will never occur pr(x wins | election today) when there is no election. It would be like me placing bets on a sports game that is not scheduled to be played.
Nate hasn't been trying to predict elections that far out, either, to be fair. He has always said some variant of "if the election were held today" when discussing the model.
Because this is what every election poll does. Nobody asks people if they would vote in such person a year from now. They always ask what is your voting preference today. If Nate Silver did something different, then he would be completely wrong.
If Nate Silver took the predictions for "if the event were to happen today" and pretended it was a prediction for the future event, he'd be wrong, but that's how most people interpret his predictions anyway so that makes him effectively wrong anyway.
Exit polls are of course asking for voting preference today, because no person in their right mind would ever answer differently for "who would you vote for today?" versus "who will you vote for on November 4?" so it doesn't matter.
The problem is, we don't care about what would happen if the election were to be run today. We care about what will happen when the election actually occurs. What we want is a prediction that takes the polls into account but also factors in all the other information we have that could be used to try and predict the future election (e.g. trying to identify whether changes in a candidates polling numbers are likely to be temporary or reflect a permanent shift, identifying historical trends that can be used to predict future behavior, etc). I realize that predicting the future is much harder than predicting the hypothetical of what would happen today, but predicting the future is the whole point of forecasting.
There is value in being able to tease out the "accurate" voter sentiments today from the various polls and accounting for uncertainty and error, but that value is primarily in how we can then turn around and use that information to aid in future forecasting, and how candidates can use that data to gauge the effects of what they're doing. But without an attempt at forecasting the future, this is largely useless to the general population.
> that's how most people interpret his predictions anyway so that makes him effectively wrong anyway.
The only thing these people need is to learn what he's really doing, and then he will be immediately correct again! To me this seems like the simple and obvious solution to this debate.
That's just not true. In 2016 they had several versions of the model. One was called the NowCast that very explicitly modeled the probability of winning the election today. They had a separate model that gave the probability of winning on election day, which included whatever their estimate was of uncertainty moving forward (including the probability of game-changing events, which is pretty low).
I think overall 538 did a pretty good job of communicating a difficult subject to a lay audience, but agree this point could perhaps be communicated better.
I don't know of a good way to try and capture these sort of events meaningfully, do you?
But that's just nonsense. If something happened yesterday that has people all hot and bothered today but which nobody will give a shit about anymore by the time the next election rolls around, Silver should say "nobody will give a shit about this anymore by the time next election rolls around." Giving a numerical value of how it would effect the election if it were to happen tomorrow is gibberish on the level of "if my grandmother had wheels she'd be a wagon".
If you think someone is offering a choice of a hamburger or a shit sandwich, you can just choose the hamburger and stop complaining about the stuff on the menu you don't like.
Ok. What were your opinions of the actual article we're commenting on? It's written by a Berkeley Math PhD, so I think it's unlikely that it includes a lot of misunderstandings of basic mathematical terms.
I've found twitter not to be a good forum for this kind of thing. Leaving the current pair alone, I've had a few times where more public figures can't determine easily that you're not one of the trolls, that and the fact that they almost certainly don't follow you and have an understanding of your context and goodwill.
For the most part, I've stopped following anyone that doesn't follow me in return on twitter.
Taking these conversations to other platforms where possible as well (e.g. Slack/Telegram groups).
Is there any reason to believe that some other medium won't have this exact problem? For instance, if you were to meet a public figure at a public event, do you think they will behave any differently? To me it seems like this is more a problem of public figures having a high number of people who want to talk to them or troll them, than of Twitter as a medium.
I mean he's not a researcher trying to predict the election and somehow has to explain everythong the layman. So some derivation from the mathmatical definitions of probability are expected, i think (the mathematical definitions in its full glory are suprisingly complicated).
I always like FiveTheirghtyEight, mostly because the plots are so beautiful and it never struck me as blatantly wrong.
In many industries if we redefined probability it would be considered malpractice or fraud.
We expect our doctors not to tell us we have a good chance of surviving surgery when we only have a 20% chance of surviving.
I would be fraud (I assume) for a lottery to tell someone buying a ticket that they have a good chance of winning millions (0000000.1% probability that you will win but a 75% probability that it will be won tonight by someone ).
i thought more of a technical argument, if this comment is based on stuff like "Event A and not A are both extremely likely".
I also think that both of your examples are not really useful, they don't violate the laws of probability. What a good chance is depends on more things than just the raw probabilities, for example i could phrase a ticket with a good chance as E(reward - price) > 0 (so its independent of its actual probability).
I don't consider "Event A and not A are both extremely likely" as P(A) != 0.5, communicating probabilities is not a mathematical problem, especially if words like "extremely" are used. Maybe P(A)=0.3 might also be extremely likely for your target audience.
Using your argument...what's a good chance of dying? It depends on the interpretation of the layperson and needs carful wording.
I remain faithful that his actual models are sound.
Actual probabilities are uncertain. If you think of the probability distribution on probabilities (the Bayesian prior), the claim that "one of the extreme probabilities is more likely than a halfway one" makes sense, eg. [0]
It makes more sense philosophically to make claims about one's model than about the real world, anyway.
"I don't consider "Event A and not A are both extremely likely"
To me, a 20% probability is not "extremely likely", but it is "extremely possible". A 10% probability is "quite possible" or "not terribly surprising". An 80% probability is "very likely", but perhaps not "extremely likely".
Piketty's extensive data on global wages raises serious questions about Pinker's narrative. And if there is still doubt Pinker's data sources [1] and interpretation are questionable to say the least.
Here is Pinker himself admitting his selective use of data [2] Watch his answer at 4.30. This is extremely poor use of stats, and makes him look a side character apologist in Hunger Games.
And here is an even more substantial criticism of Pinker's approach by Leon Wieseltier in the newrepublic [3].
Taleb inexplicably chose this as the hill to die on [2] and wrote a lengthy and technical paper to refute, and replied (on the [2] thread) "When someone says are event and its opposite are extremely possible I infer either 1) 50/50 or 2) the predictor is shamelessly hedging, in other words, BS."
I've often found Taleb's ideas to be profound and interesting, and unfortunately offset by his arrogance and occasionally cryptic pronouncements and colorful colloquial "Brooklyn-ese" intentional misspellings of words (like "gabish" for "capisce"). But in this case I'm having trouble following what his criticism even is -- forecasts, especially probability forecasts, are hard, and evaluating them is hard -- the dimensions of skill vs. calibration vs. precision are hard to evaluate, and Nate Silver does a fairly good job of describing the tension.
[1] https://twitter.com/DineshDSouza/status/1059147114826162177
[2] https://twitter.com/nntaleb/status/1059202026184282113