3
$\begingroup$

It is said that there is no such thing as a truly uninformative prior.

For example, here.

Q:

  1. Has it been proven that a truly uninformative prior does not exist, or is it merely the case that such a prior has not been found?

  2. If it has been proven, then what is the conceptual explanation of why a truly uninformative prior cannot exist?

Improve this question
$\endgroup$
6
  • 3
    $\begingroup$ Is this truly a different question from the one that you are linking to? $\endgroup$
    – Sextus Empiricus
    Commented Jun 10 at 7:03
  • 1
    $\begingroup$ What do you mean by 'truly uninformative'? Are you asking, say, for a proof that reparametrization can always unflatten a uniform prior (w.r.t. the 1st paragraph of Xian's answer)? $\endgroup$
    – Scortchi ♦
    Commented Jun 10 at 13:20
  • $\begingroup$ The 2nd paragraph of Xian's answer in the related thread basically says that 'uninformative' in a sense of maximum entropy will still require constraints for the distribution whose entropy is maximises, and also the base measure playes an influence (in a similar way as the uniform prior elbeing related tonthe base measure). So wlitnwill always depend on certain subjective choices, and 'information' is relative to that background. $\endgroup$
    – Sextus Empiricus
    Commented Jun 10 at 14:05
  • $\begingroup$ @SextusEmpiricus and Scortchi - Reinstate Monica: I had given a close vote for the same reason earlier but then decided that answers to the linked question don't address the question of mathematical proof, which seems central to the current question. So I have taken back my close vote. $\endgroup$
    – Christian Hennig
    Commented Jun 10 at 14:50
  • 1
    $\begingroup$ @Scortchi I wonder whether it is much worth to research this. Without a clear mathematical definition of uninformative, there is nothing to prove (at least not mathematically). $\endgroup$
    – Sextus Empiricus
    Commented Jun 10 at 16:33

1 Answer 1

Reset to default
10
$\begingroup$

First, answers in this thread (already linked in the question) are very informative, and much that can be said is there.

I just add the rather "metastatistical" consideration that any possible "proof" would have to depend on a definition of what "truly uninformative" actually means, and there is no generally accepted such definition. If you look at the linked thread but also this paper (Kass, R. E., & Wasserman, L. (1996). The Selection of Prior Distributions by Formal Rules. Journal of the American Statistical Association, 91(435), 1343–1370. https://doi.org/10.1080/01621459.1996.10477003; freely available in many places), a good number of such definitions have been proposed. As long as you accept one of these definitions, you can find "uninformative" priors in many situations, however the general argument is that something that is "uninformative" according to one definition is informative according to another. In particular one can have good arguments that any such definition doesn't exclude everything that we would reasonably call "informative" in non-mathematical terms, and therefore no existing definition qualifies as actually defining "truly uninformative".

The thing here is that if you wanted to prove that "a truly uninformative prior doesn't exist", you'd need a mathematical definition of "truly uninformative", and this doesn't exist because there is no agreement (for good reasons) that any existing definition of "uninformative in some sense" could deliver such a thing.

Maybe the simplest argument would be that whatever is implied by a prior (and be it uniformity) is some kind of information. You may not be happy with that, but any attempt at a mathematical definition of "uninformative" doesn't make this very simple consideration go away.

Improve this answer
$\endgroup$
1
  • $\begingroup$ (+1) It usually makes sense to parse an "uninformative prior" as a "minimally informative prior" in one sense or another. $\endgroup$
    – Scortchi ♦
    Commented Jun 11 at 7:34

Not the answer you're looking for? Browse other questions tagged or ask your own question.