Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Inspired by a comment from this question:

What do we consider "uninformative" in a prior - and what information is still contained in a supposedly uninformative prior?

I generally see the prior in an analysis where it's either a frequentist-type analysis trying to borrow some nice parts from Bayesian analysis (be it some easier interpretation all the way to 'its the hot thing to do'), the specified prior is a uniform distribution across the bounds of the effect measure, centered on 0. But even that asserts a shape to the prior - it just happens to be flat.

Is there a better uninformative prior to use?

share|improve this question
    
Maybe you’ll enjoy a look on the so-called Principle of Maximum Entropy. I don’t feel like expanding that in a full answer – the Wikipedia article seems of good quality. I’m pretty confident some contributors will expand on it that much better than I would. –  Elvis Jan 3 '12 at 9:54

2 Answers 2

up vote 22 down vote accepted

The "uninformative" prior is sadly a misnomer. Any prior distribution contains some specification that is akin to some amount of information. Even the uniform prior. Indeed, the uniform prior is only flat for one parameterisation of the problem. If one changes to another parameterisation (even a bounded one), the Jacobian comes into play and the prior is flat no longer.

As pointed out by Elvis, maximum entropy is one approach to select so-called "uninformative" priors. It however requires (a) some level of information on some moments of the prior to specify the condtraints and (b) the choice of a reference measure in continuous settings, which brings the debate back to its initial stage!

José Bernardo has produced an original theory of reference priors where he chooses the prior in order to maximise the information brought by the data. In the simplest cases, the solution is Jeffreys' prior. In more complex problems, (a) a choice of the parameters of interest must be made; (b) the computation of the prior is fairly involved. (See e.g. my book The Bayesian Choice for details.)

In short, there is no "best" (or even "better") choice for "the" "uninformative" prior. And I think there should not be because the very nature of Bayesian analysis implies that the choice of the prior distribution matters. And that there is no comparison of priors: one cannot be "better" than another. The conclusion of José Bernardo, Jim Berger and many other "objective" Bayesians is that there are roughly equivalent reference priors one can use when unsure about one's prior information, some being backed up by information theory arguments, others by non-Bayesian frequentist properties, and conducting to rather similar inferences.

share|improve this answer
4  
(+1) Your book? Oh damn. I so have 387 questions for you :) –  Elvis Jan 3 '12 at 13:07
2  
(+1) For an objective (no less!), straightforward answer. –  cardinal Jan 3 '12 at 15:00
1  
+1 Thank you for a good and well-informed overview of the issues. –  whuber Jan 3 '12 at 15:00
1  
An outstanding answer. Thank you. And yet another book to go on the wish list. –  Fomite Jan 3 '12 at 18:26
1  
It's almost unfair. After all, he's Christian Robert! Just kidding. Great answer. And I'd love if @Xi'an could expand it in a post at his blog, specially about how parametrization is important to the topic of "uninformative" priors. –  Manoel Galdino Jan 13 '12 at 19:35

An appealing property of formal noninformative priors is the "frequentist-matching property" : it means that a posterior 95%-credibility interval is also (at least, approximately) a 95%-confidence interval in the frequentist sense. This property holds for Bernardo's reference prior although the fundations of these noninformative priors are not oriented towards the achievement of a good frequentist-matching property, If you use a "naive" ("flat") noninformative prior such as the uniform distribution or a Gaussian distribution with a huge variance then there is no guarantee that the frequentist-matching property holds. Maybe Bernardo's reference prior could not be considered as the "best" choice of a noninformative prior but could be considered as the most successful one. Theoretically it overcomes many paradoxes of other candidates.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.