I've recently embarked on fitting regression mixed models in the Bayesian framework, using a MCMC algorithm (function MCMCglmm in R actually).

I believe I have understood how to diagnose convergence of the estimation process (trace, geweke plot, autocorrelation, posterior distribution...).

One of the thing that strikes me in the Bayesian framework is that much effort seems to devoted to do those diagnostics, whereas very little appears to be done in terms of checking the residuals of the fitted model. For instance in MCMCglmm the residual.mcmc() function does exist but is actually not yet implemented (ie.returns: "residuals not yet implemented for MCMCglmm objects"; same story for predict.mcmc()). It seems to be lacking from other packages too, and more generally is little discussed in the literature I've found (apart from DIC which is quite heavily discussed too).

Could anyone point me to some useful references, and ideally R code I could play with or modify?

Many thanks.

  • $\begingroup$ Great question. I really like Andrew Gelman's paper with Cosma Shalizi about Bayesian model checking. $\endgroup$ – David J. Harris Sep 5 '13 at 18:32

I think the use of the term residual is not consistent with Bayesian regression. Remember, in frequentist probability models, it's the parameters which are considered fixed estimable quantities and the data generating mechanism has some random probability model associated with observed data. For Bayesians, the parameters of probability models are considered to be variable and the fixed data update our belief about what those parameters are. Therefore, if you were calculating the variance of the observed minus fitted values in a regression model, the observed component would have 0 variance whereas the fitted component would vary as a function of the posterior probability density for the model parameters. This is the opposite of what you would derive from the frequentist regression model. I think if one were interested in checking the probabilistic assumptions of their Bayesian regression model, a simple QQplot of the posterior density of parameter estimates (estimated from our MCMC sampling) versus a normal distribution would have diagnostic power analogous to analyzing residuals (or Pearson residuals for non-linear link functions).

  • 1
    $\begingroup$ This is a good answer. There may yet be answers that give useful Bayesian constructs calculated from the observed-minus-fitted residual, but this one certainly shouldn't have been downvoted. $\endgroup$ – ely Sep 5 '13 at 21:16
  • 3
    $\begingroup$ Also, it might be worth clarifying that in the Bayesian setting you don't really have "fitted" values. You could calculate the posterior mean for a given observed input, to get the maximum a posteriori estimate of the expected value of the target variable at that input. But this would be reducing everything to point estimates, which is not usually desired if you're doing Bayesian inference. $\endgroup$ – ely Sep 5 '13 at 21:19
  • 2
    $\begingroup$ @EMS any of those are meaningful residuals. Just because one is a Bayesian doesn't mean one can't check whether the assumptions are reflected in the data. $\endgroup$ – Glen_b Sep 6 '13 at 2:01
  • 1
    $\begingroup$ For exact probabilistic inference (normality assumptions in place) in the frequentist setting, "residuals" would, in replications of the study experiment, be conditionally independent of the "fitted value" (or conditional mean). In the Bayes world, data aren't random, so what would be conditionally independent of what? $\endgroup$ – AdamO Sep 6 '13 at 20:04
  • 1
    $\begingroup$ Regression in Bayesian and exact frequentist procedures is a way of estimating the conditional mean $\mbox{E}[Y|X]$, usually described by a set of model parameters. I am using the conditional mean to refer to fitted values. In frequentist statistics, there is a concept of repeated experiments, although we conceive of the sample size and distribution of $X$ as being fixed whereas $Y$ is random and subject to replications through repeated experimentation. That's the frequentist interpretation of probability. That's why residuals have properties of random variables, notably a density function. $\endgroup$ – AdamO Sep 6 '13 at 21:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.