Bayesian and frequentist interpretations vs approaches

Question

I have been reading about the frequentist vs bayesian issue (this article has helped a lot, specially with the example; also this one), and I haven't come to terms with it. At the moment it seems like there are the frequentist and bayesian interpretation of probability; and, separately, the frequentist and bayesian approach to problems. The former is about the belief vs frequency issue (illustrated in the second article). The later is illustrated in the first article. Both put together seem to me like this:

• The frequentist interpretation of the frequentist approach ensures to be right a% of the time for large number of trials assuming only the likelihood distribution, no matter which parameter we get, as long we as assume that we'll get a good range of data.
• The frequentist interpretation of the bayesian approach ensures to be right a% of the time for large number of trials assuming the likelihood distribution and the prior, no matter which data we get, as long as we assume that we'll get a good range of parameters.
• The bayesian interpretation of the frequentist approach says that we are right with a probability of a% assuming only the likelihood distribution, no matter which parameter we get, as long as we assume fairness in the randomness of the data.
• The bayesian interpretation of the bayesian approach says that we are right with a probability of a% assuming the likelihood distribution and the prior, no matter which data we get, as long as we assume fairness in the randomness of the parameters.

This is the only consistent view that I have been able to form from what I've read. However, I I still think I maybe missing something (as I actually haven't found this view like this anywhere else, it's my own conclusion), so taking the null hypothesis that I'm wrong, where's my mistake?

Answer 1

Your notes about frequentists relying on "repeated sampling" properites and Bayesians relying on "fairness" is in line with Neyman/Pearson and deFinetti's justifications of each paradigm, respectively. Bayesian and frequentist approaches are appropriate in different contexts. A controversial aspect of frequentist approaches is the relevance of the concept of "confidence" in the case where it is not clear what is the "embedding series" of experiements (there can be many, look up "relevant subsets problem" for more on this). Bayesians get critisized for applying priors when the underlying property is not random...hence there is a "calibration" problem with a, say, 95% posterior interval...95% of what, and why do we care?

I'd take a look at another school/paradigm as well...the Likelihood school, as described by the accessible and useful book "In All Likelihood" by Yudi Pawitan. This approach shows how the objective and subjective aspects of probability are related via the distinction between likelihood and probability.

Also, there is an interesting "meeting of the minds" when it comes to random-effects modeling. Take a look at that to see how the two, in practice, can converge in concepts.

Answer 2

I think I may have found the problem with my argument, i.e. how what i called "the frequentist interpretation of the bayesian approach" and viceversa don't really make sense:

The "frequentist interpretation of the bayesian approach", as described in my quesion, doesn't make much sense because it says that it assumes the likelihood function (and the prior), but then says "no matter which data we get [in the hypothetical large set of experiments]" which is incompatible with the frequentist interpretation of the likelihood function.

The "bayesian interpretation of the frequentist approach", is also wrong because it doesn't ensure what I say below. For example, in the frequentist approach, I may well make a measurement and have an emtpy confidence interval for it, which clearly means that it doesn't ensure an a% probability of being right.

So I think I understand it better now. And so it seems that if you want to be as cautious as possible, frequentist is the way to go. But if you don't have much statistical significance, or have good reasons for your prior, bayesian is the way to go.