Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

To compare two Bayesian models for the same data, I plotted the fitted values of the first model versus the fitted values of the second model. The plot indicates a perfect similarity.

Relying on this plot, I said there is no significant difference between these two models. However, the difference between the Deviance information criterion (DIC) values of each model is 85, which I think shows that there is a significant difference between these two models.

So, the plot and the DIC, I suppose, are in contrast. What does everyone else think?

share|improve this question
1  
Is this not just another version of the distinction between statistical significance & practical significance? –  Scortchi Jan 14 '13 at 15:08
    
I don't see this connection. I think the poster is genuinely confused about how to make formal inference between two models and for what purpose. –  AdamO Jan 14 '13 at 16:08
add comment

1 Answer

The DIC goes further than just the fitted values for the data, but actually uses the probability model for them as the wiki article suggests. Details notwithstanding, I'd bet the probability models for the outcome differ between the models you've fit.

As an example, suppose I have aggregate binary proportions across a range of continuous age values. I could fit a linear model and I could fit a GLM with binomial variance. Both will give me the exact same fitted values and singular confidence intervals for the model coefficients. However, the implicit probability models for these data are different.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.