I recently finished reading The Lady Tasting Tea, a fun book about the history of statistics. At the end of the book, the author, David Salsburg, proposes three open philosophical problems in statistics, the solutions to which he argues would have larger implications for the application of statistical theory to science. I had never heard of these problems before, so I am interested in other people's reactions to them. I am venturing into territory about which I have little knowledge, so I'm just going to describe Salsburg's portrayal of these problems and pose two general questions about these problems below.
Salsburg's philosophical problems are:
- Can statistical models be used to make decisions?
- What is the meaning of probability when applied to real life?
- Do people really understand probability?
Statistics and decision making
As an illustration of the problem presented in question 1, Salsburg presents the following paradox. Suppose we organize a lottery with 10000 unnumbered tickets. If we use probability to make a decision about whether any given ticket will win the lottery by rejecting this hypothesis for tickets with probabilities below, say, .001, we will reject the hypothesis of a winning ticket for all the tickets in the lottery!
Salsburg uses this example to argue that logic is inconsistent with probability theory as probability theory is currently understood, and that, therefore, we currently do not have a good means of integrating statistics (which, in its modern form, is based in large part on probability theory) with a logical means of decision-making.
The meaning of probability
As a mathematical abstraction, Salsburg argues that probability works well, but when we attempt to apply the results to real life, we run into the problem that probability has no concrete meaning in real life. More specifically, when we say that there is a 95% chance of rain tomorrow, it is unclear to what entities that 95% applies. Does it apply to the set of possible experiments that we could conduct to obtain knowledge about rain? Does it apply to the set of people who might go outside and get wet? Salsburg argues that the lack of a means to interpret probabilities creates problems for any statistical model based on probability (i.e., most of them).
Do people understand probability?
Salsburg argues that one attempt to resolve the issues with the lack of a concrete means of interpreting probability is through the concept of "personal probability", proposed by Jimmie Savage and Bruno de Finetti, which understands probability as personal beliefs about the likelihood of future events. However, in order for personal probability to provide a coherent basis for probability, people need to have a common understanding of what probability is and a common means of using evidence to draw conclusions about probability. Unfortunately, evidence such as that produce by Kahneman and Tversky suggests that personal beliefs might be a difficult basis on which to create a coherent basis for probability. Salsburg suggests that statistical methods that model probabilities as beliefs (perhaps such as Bayesian methods? I'm stretching my knowledge here) will need to deal with this problem.
- To what extent are Salsburg's problems really problems for modern statistics?
- Have we made any progress towards finding resolutions to these problems?