# Why is a Bayesian not allowed to look at the residuals?

In the article "Discussion: Should Ecologists Become Bayesians?" Brian Dennis gives a surprisingly balanced and positive view of Bayesian statistics when his aim seems to be to warn people about it. However, in one paragraph, without any citations or justifications, he says:

Bayesians, you see, are not allowed to look at their residuals. It violates the likelihood principle to judge an outcome by how extreme it is under a model. To a Bayesian, there are no bad models, just bad beliefs.

Why would a Bayesian not be allowed to look at the residuals? What would be the appropriate citation for this (i.e. who is he quoting)?

Dennis, B.
Discussion: Should Ecologists Become Bayesians?
Ecological Applications, Ecological Society of America, 1996, 6, 1095-1103

-
If that argument worked, frequentists couldn't use the likelihood principle either - for the same reason. –  Glen_b Feb 6 at 9:32
@Glen: Frequentist analysis does violate the likelihood principle. –  Scortchi Feb 6 at 9:40
@Glen: A frequentist truly beholden to the LP (the weak version, equivalent to the Sufficiency Principle - the strong version is simply incompatible with the frequentist approach) would have to shun model checking. The ones who just admire it are glad when they can use it for the job of estimating the parameters of a specified model & still have more-or-less independent ancillaries - the residuals - left over for model checking any old how. –  Scortchi Feb 6 at 11:21
Even when the frequentist do ML estimation he still violates the LP because he considers the sampling distribution of the MLE to find a confidence interval for his estimate. –  Zen Feb 6 at 14:24
@Zen: He doesn't violate the weak LP as long as the confidence interval depends on the data only through the likelihood function. But he may sooner or later violate the strong LP by making a different confidence interval based on the same likelihood function from a different experiment with a different sampling space. –  Scortchi Feb 6 at 15:27

Of course Bayesians can look at the residuals! And of course there are bad models in Bayesian analysis. Maybe a few Bayesians in the 70's supported views like that (and I doubt that), but you will hardly find any Bayesian supporting this view these days.

I didn't read the text, but Bayesians use things like Bayes factors to compare models. Actually, a Bayesian can even compute the probability of a model being true and pick the model which is more likely to be true. Or a Bayesian can average across models, to achieve a better model. Or can use posterior predictive checks. There are a lot of options to check a model and each one may favor one approach or another, but to say that there are no bad models in Bayesian analysis is non-sense.

So, at most, it would be more appropriate to say that in some extreme versions of Bayesianism (extreme versions that almost no one uses in applied settings, by the way) you're not allowed to check your model. But than you could say that in some extreme versions of frequentism you're not allowed to use observational data as well. But why waste time discussing these silly things, when we can discuss if and when, in an applied setting, we should use Bayesian or frequentist methods or whatever? That's what's important, in my humble opinion.

Update: The OP asked for a reference of someone advocating the extreme version of Bayes. Since I never read any extreme version of Bayes, I can't provide this reference. But I'd guess that Savage may be such a reference. I never read anything written by him, so I may be wrong.

ps.: Think about the problem of the "well-calibrated Bayesian" (Dawid (1982), JASA, 77, 379). A coherent subjectivist Bayesian forecaster can't be uncalibrated, and so wouldn't review his model/forecasts despite any overwhelming evidence that he's uncalibrated. But I don't think anyone in practice can claim to be that coherent. Thus, model review is important.

ps2.: I like this paper by Efron as well. The full reference is: Efron, Bradley (2005). "Bayesians, frequentists, and scientists." Journal of the American Statistical Association 100(469).

-
I also assumed that the prohibition was never taken seriously in practice, so I was surprised to read this from Gelman:- " I certainly don’t want to return to the circa-1990 status quo in Bayesian statistics, in which it was considered virtually illegal to check your model’s fit to data." –  Scortchi Feb 6 at 12:36
I don't know how was Bayesian statistics in the nineties. But it's hard to believe that in applied settings Bayesians didn't check their models. Maybe they checked, but didn't tell! –  Manoel Galdino Feb 6 at 12:53
You're probably right: illegal$\neq$unusual. Perhaps it was more common back then, at least among academics, to defend Bayesian methods in principle rather than pointing to evident successes in application, & therefore any non-conformity to (real or supposed) principles would get swept under the rug. –  Scortchi Feb 6 at 13:02
I definitely agree that this is not a major issue, I was just curious if anybody had published on this. Have you ever read anybody advocating these "extreme versions of Bayesianism"? –  Mankka Feb 6 at 13:30