Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I've recently embarked on fitting regression mixed models in the Bayesian framework, using a MCMC algorithm (function MCMCglmm in R actually).

I believe I have understood how to diagnose convergence of the estimation process (trace, geweke plot, autocorrelation, posterior distribution...).

One of the thing that strikes me in the Bayesian framework is that much effort seems to devoted to do those diagnostics, whereas very little appears to be done in terms of checking the residuals of the fitted model. For instance in MCMCglmm the residual.mcmc() function does exist but is actually not yet implemented (ie.returns: "residuals not yet implemented for MCMCglmm objects"; same story for predict.mcmc()). It seems to be lacking from other packages too, and more generally is little discussed in the literature I've found (apart from DIC which is quite heavily discussed too).

Could anyone point me to some useful references, and ideally R code I could play with or modify?

Many thanks.

share|improve this question
    
Great question. I really like Andrew Gelman's paper with Cosma Shalizi about Bayesian model checking. –  David J. Harris Sep 5 '13 at 18:32
add comment

1 Answer 1

I think the use of the term residual is not consistent with Bayesian regression. Remember, in frequentist probability models, it's the parameters which are considered fixed estimable quantities and the data generating mechanism has some random probability model associated with observed data. For Bayesians, the parameters of probability models are considered to be variable and the fixed data update our belief about what those parameters are. Therefore, if you were calculating the variance of the observed minus fitted values in a regression model, the observed component would have 0 variance whereas the fitted component would vary as a function of the posterior probability density for the model parameters. This is the opposite of what you would derive from the frequentist regression model. I think if one were interested in checking the probabilistic assumptions of their Bayesian regression model, a simple QQplot of the posterior density of parameter estimates (estimated from our MCMC sampling) versus a normal distribution would have diagnostic power analogous to analyzing residuals (or Pearson residuals for non-linear link functions).

share|improve this answer
1  
This is a good answer. There may yet be answers that give useful Bayesian constructs calculated from the observed-minus-fitted residual, but this one certainly shouldn't have been downvoted. –  EMS Sep 5 '13 at 21:16
2  
Also, it might be worth clarifying that in the Bayesian setting you don't really have "fitted" values. You could calculate the posterior mean for a given observed input, to get the maximum a posteriori estimate of the expected value of the target variable at that input. But this would be reducing everything to point estimates, which is not usually desired if you're doing Bayesian inference. –  EMS Sep 5 '13 at 21:19
    
This is why "residual" is less meanginful. Residual from what? The posterior mean value? The posterior mode value? The posterior's 42.763% quantile value? What makes any one of those a better thing to use as "the" estimate than any other? –  EMS Sep 5 '13 at 21:20
    
Yes, good point and thanks for the clarification on my understanding as well. –  AdamO Sep 5 '13 at 21:35
2  
@EMS any of those are meaningful residuals. Just because one is a Bayesian doesn't mean one can't check whether the assumptions are reflected in the data. –  Glen_b Sep 6 '13 at 2:01
show 8 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.