How important are interpretations of probability to the practice of statistics?

Question

I know that the frequentist interpretation of probability is associated with classical statistics and maximum likelihood estimation, and that a subjective interpretation of probability is considered integral to Bayesian statistics. But are these interpretations of probability essential to the statistical methods, or are they historical? Specifically, what is the connection between the frequentist interpretation of probability and maximum likelihood estimation? Also, is there any reason why someone who subscribes to the "propensity" interpretation of probability couldn't use either approach, or favor one over the other? From my understanding, it seems like the most important conceptual difference is between considering parameters fixed and unknown (classical), and modelling parameters with probability distributions (Bayesian). But is there some reason why a person who believed in a frequentist interpretation of probability couldn't say "even though the parameter is fixed, I will model the range of its plausible values with a probability distribution" and do Bayesian statistics to arrive at an estimate?

Answer 1

I would highly recommend you read the works of Deborah Mayo and Andrew Gelman.enter link description here. Dr. Mayo represents a modern view of Frequentism-style inference while Dr. Gelman's is very well known expositor of modern Bayesian approaches.

Dr. Gelman has a nice piece where he discusses how modern Bayesian statistics and modern Frequentist statistics share a common concern with the "calibration" of a model's inferences and predictions. In other words, if a Bayesian makes 1000 predictions, each with a posterior probability of 40% (of being correct), then they would be worried if only 200 were correct, and intrigued (but also concerned) if 900 were correct. They'd expect around 400.

Similarly, as explained by Mayo and in Gelman's paper, frequentist statistical techniques are concerned with long-run performance over repeated trials: a 95% confidence interval should cover the true parameter in 95% of samples.

It should be noted that the area of "random effects models" is a really nice bridge between the Bayesian and Frequentist methodological worlds. Here is a place where even frequentists allow random parameters. Bayesian's simply treat every problem as fundamentally a random effects model.

Please bear in mind that neither the Bayesian nor the Frequentist knows if their model is correct (actually, most are almost sure their models are wrong), so in either case, the models are really just tools that provide an interpretative framework for some real-world phenomenon. So, we get to reduce a complex process to a discussion about $\gamma,\mu,$ and other parameters, instead of having to to tackle the whole mess. It may not be physically/mechanistically correct, but nonetheless, its science works, especially in the social sciences (e.g., economics). This "instrumentalism" not a problem insofar as we can verify our models against data and check their calibration.

So, both schools of statistics ultimately rest on a "frequency" interpretation of probability, since it is the only one that has any hope of experimental confirmation (if only partially). However, they differ in the types of models used and the "reference set" over which we calculate our probability of error: Fequentists use "fixed effects" models and calibrate them against repetitions of the same experiment with the parameters held constant. Bayesians prefer "random effects" models and calibrate their errors over all possible realizations of the random effects (i.e., the possible underlying models) while holding observations constant.

In the end, a modern statistician's views of a model's value should be based primarily how well its predictions match observations, not on philosophical/ideological issues.