# How do I interpret the credibility interval in a Bayesian Regularized Regression?

A penalized regression provides biased estimates of the regression coefficients (bias-variance trade-off principle). Therefore, standard errors and confidence intervals are regarded as not very meaningful for those biased estimates arising from (frequentist) penalized regression method, see e.g. the discussion Estimating R-squared and statistical significance from penalized regression model . I would assume that the same problems exists in an Bayesian approach but Kyung, Gill, Ghaosh and Casella (2010) say that the Bayesian formulation produces valid standard errors. Does it mean that a 95% credibility intervals includes with 95% probability the true biased estimate and if yes, is this a useful information?

• Well, the interpretation would rather be that a 95% credibility intervals includes with 95% probability the true parameter (given the model assuptions, of course...). I would say that the 95% credibility interval is extremely useful information, and often the reported end result of a Bayesian analysis (If you do not choose to look at the complete posterior distribution, that is). Commented Oct 17, 2013 at 9:07
• Thanks a lot Rasmus, that's the answer I had hoped for... Do you know an article where it is explained? Commented Oct 23, 2013 at 8:10
• It is very well explained in John Kruschkes book, Doing Bayesian Data Analysis. indiana.edu/~kruschke/DoingBayesianDataAnalysis Commented Oct 23, 2013 at 12:41
• Thanks for your reference, Rasmus but sorry but you may misunderstood my question. I wanted to know if the credibility interval around a biased estimate of a penalized regression can be interpreted as the credibility interval for the true estimate or for the biased estiamte. If it is a credibility interval for a biased estiamte is it really a useful information? Commented Nov 26, 2013 at 13:25
• Yes. But still, that is given your model and your assumptions. And I believe you will never get away from the "given your model and your assumptions" part... A nice article regarding the bias/variance tradeoff can be found here: demotrends.wordpress.com/2013/09/04/… Commented Nov 26, 2013 at 20:01