… The forecasts were off. We were forecasting Biden to get 54.4% of the two-party vote and it seems that he only got 52% or so. We forecasted Biden at 356 electoral votes and it seems that he’ll only end up with 280 or so. We had uncertainty intervals, and it looks like the outcome will fall within those intervals, but, still, we can’t be so happy about having issued that 96% win probability. Our model messed up.
But, here’s the thing. Suppose we’d included wider uncertainty intervals so the outcome was, say, within the 50% predictive interval. Fine. If we’d given Biden a 75% chance of winning and then he wins by a narrow margin, the forecast would look just fine and I’d be happier with our model. But the polls would still have messed up, it’s just that we would’ve better included the possibility of messing up in our model.
To put it another way: a statement such as “The polls messed up,” is not just a statement about the polls, it’s a statement about how the polls are interpreted. CONT.
Andrew Gelman (Columbia U.), Statistical Modeling, Causal Inference, and Social Science