Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

It is my understanding that we should not be allowing the same data set we are analyzing to drive/define what the prior distributions look like in a Bayesian analysis. Specifically, it is inappropriate to define prior distributions for a Bayesian analysis based on summary statistics from the same data set that you are then going to use the priors to help fit a model to.

Does anyone happen to know of resources that specifically discuss this as being inappropriate? I need some citations for this issue.

share|improve this question
    

2 Answers 2

Yes this is inappropriate because it uses the same data twice, leading to falsely overconfident results. This is known as 'double dipping'.

For references, I would start with Carlin and Louis (2000). Although 'double dipping' has been one of the primary critiques of Empirical Bayes, Ch. 3, in particular section 3.5, of this book describes ways to estimate appropriate confidence intervals using the EB approach.

Berger J (2006). \The Case for Objective Bayesian Analysis." Bayesian Analysis, 1(3), 385{ 402

Bradley P. Carlin, Thomas A. Louis 2000. Bayes and Empirical Bayes methods for data analysis.

Darniede, W.F. 2011. Bayesian Methods for Data-Dependent Priors. MS Thesis, Ohio State Univ.

Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. (2003), Bayesian Data Anal- ysis, Second Edition (Chapman & Hall/CRC Texts in Statistical Science), Chap- man and Hall/CRC, 2nd ed.

share|improve this answer
    
Thank you! This is very helpful! –  sarah Sep 22 '11 at 19:16
    
@sarah Please register your account so you could reclaim your question. Just visit this url: stats.stackexchange.com/users/login –  mbq Sep 22 '11 at 19:19

It can make sense to use the data to build the prior though.

For an example in mixture modelling, see Richardson & Green (1997): http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.27.3667

They use the mean and the range of the data points as hyperparameters for the prior and it makes perfect sense.

The problem of using the data twice occurs when a informative prior is derived from the data, in my opinion.

As long as you check that your prior distribution is "flat" where the posterior distribution is peaked, then you know that your prior distribution has not a strong impact on the results.

share|improve this answer
    
Using the data to build the prior cannot take place within the Bayesian paradigm. So it does not make sense from a Bayesian perspective and the usual validation of Bayesian procedures does not apply. The resulting inference may be perfectly valid but one has to demonstrate it from first principles. (Richardson and Green use what is called empirical Bayes. Which is not a Bayesian procedure.) –  Xi'an Jan 9 '12 at 11:20

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.