In my mind, (I think) I understand the difference between prediction and uncertainty about that prediction. The prediction comes from a model (say a LPM or a Probit) and the uncertainty is related to how wide the confidence intervals are around a particular prediction. However I am having a hard time explaining to a friend that prediction is different from uncertainty, in particular when the prediction is about a probability. For example, my friend is interested in sports and he posits that if you (ie a model) predicts that Team A will win over Team B with 95% chance, it should be very unlikely that Team B ends up winning. He is frustrated when Team B wins, and I tell him that (1) there still was a 5% chance of Team B winning, but also (2) that there could have been a lot of uncertainty around that 95% prediction of A winning. He replies that if there is a lot of uncertainty in the prediction, then it should not have been 95% in the first place. This is when our conversation stalls. I keep repeating that prediction is not the same as uncertainty, while he keeps digging on "then the probability of A winning should not have been so high." How to best explain the difference to my friend?
Also, probably we are falling into some fallacy, and if so I'd like to know which one.
This issue is also probably related to uncertainty of the model vs uncertainty of the prediction. Where can I find some reading material about that?