Three open philosophical problems in statistics

Question

I recently finished reading The Lady Tasting Tea, a fun book about the history of statistics. At the end of the book, the author, David Salsburg, proposes three open philosophical problems in statistics, the solutions to which he argues would have larger implications for the application of statistical theory to science. I had never heard of these problems before, so I am interested in other people's reactions to them. I am venturing into territory about which I have little knowledge, so I'm just going to describe Salsburg's portrayal of these problems and pose two general questions about these problems below.

Salsburg's philosophical problems are:

1. Can statistical models be used to make decisions?
2. What is the meaning of probability when applied to real life?
3. Do people really understand probability?

Statistics and decision making

As an illustration of the problem presented in question 1, Salsburg presents the following paradox. Suppose we organize a lottery with 10000 unnumbered tickets. If we use probability to make a decision about whether any given ticket will win the lottery by rejecting this hypothesis for tickets with probabilities below, say, .001, we will reject the hypothesis of a winning ticket for all the tickets in the lottery!

Salsburg uses this example to argue that logic is inconsistent with probability theory as probability theory is currently understood, and that, therefore, we currently do not have a good means of integrating statistics (which, in its modern form, is based in large part on probability theory) with a logical means of decision-making.

The meaning of probability

As a mathematical abstraction, Salsburg argues that probability works well, but when we attempt to apply the results to real life, we run into the problem that probability has no concrete meaning in real life. More specifically, when we say that there is a 95% chance of rain tomorrow, it is unclear to what entities that 95% applies. Does it apply to the set of possible experiments that we could conduct to obtain knowledge about rain? Does it apply to the set of people who might go outside and get wet? Salsburg argues that the lack of a means to interpret probabilities creates problems for any statistical model based on probability (i.e., most of them).

Do people understand probability?

Salsburg argues that one attempt to resolve the issues with the lack of a concrete means of interpreting probability is through the concept of "personal probability", proposed by Jimmie Savage and Bruno de Finetti, which understands probability as personal beliefs about the likelihood of future events. However, in order for personal probability to provide a coherent basis for probability, people need to have a common understanding of what probability is and a common means of using evidence to draw conclusions about probability. Unfortunately, evidence such as that produce by Kahneman and Tversky suggests that personal beliefs might be a difficult basis on which to create a coherent basis for probability. Salsburg suggests that statistical methods that model probabilities as beliefs (perhaps such as Bayesian methods? I'm stretching my knowledge here) will need to deal with this problem.

My questions

1. To what extent are Salsburg's problems really problems for modern statistics?
2. Have we made any progress towards finding resolutions to these problems?

Answer 1

I don't think these really are questions which can be answered conclusively. (IOW, they are, indeed, philosophical). That said...

Statistics and decision making

Yes, we can use statistics in decision making.

However, there are limits to its applicability; IOW, one has to understand what one is doing.

This is fully applicable to any theory.

The meaning of probability

95% probability of rain tomorrow means that if your cost of preparing for a rain (e.g., taking the umbrella) is A and your cost of being caught in the rain unprepared (e.g., wet suit) is B, then you should take the umbrella with you iff A < 0.95 * B.

Do people understand probability?

No, people do not understand much, least of all probability.

Kahneman and Tversky have shown that human intuition is flawed on many levels, but intuition and understanding are not identical, and I would argue that people understand even less than they intuit.

To what extent are Salsburg's problems really problems for modern statistics?

Nil. I don't think anyone cares about these issues except for philosophers and those in a philosophical mood.

Have we made any progress towards finding resolutions to these problems?

Everyone who cares has a resolution. My personal resolution is above.

Answer 2

Can we use statistics/probability to make decisions? Of course we can, the way in which we should go about this is by choosing the course of action that minimises our expected loss. In this case, all lottery numbers are equally likely to come up; if all provide the same prize, then the expected loss is the same for any number, so it doesn't matter which we choose. If we also have the option not to play the lottery, that would probably be the course of action we should take as it will minimise our expected loss assuming that the lottery makes a profit for somebody (or at least covers the cost of running the lottery). Of course this is just common sense and is consistent with logic, and could be expressed in purely probabilistic terms.

It seems to me that the question arises from a rather limited view of how statistics can be used to make decisions, it doesn't have to be done with quasi-Fisherian hypothesis tests.

I would suggest that Jaynes book on Probability theory goes a fair way to addressing points (2) and (3), probabilities can represent objective measures of plausibility without them being "personal probabilities", but I expect @probabilityislogic can explain that better than I can.