When using the "non-informative" prior $\pi(\mu,\sigma)\propto\frac{1}{\sigma^2}$ where $\pi(\mu)\propto1$ and $\pi(\sigma^2)\propto\frac{1}{\sigma^2}$

Where is the no information for the parameter?

and if would be informative prior, Where one would see the 'information' given?

What I've searched

I saw the definition on Wikipedia https://en.wikipedia.org/wiki/Prior_probability and there mentions

  • An informative prior expresses specific, definite information about a variable. (then an example that I didn't understand).

  • An uninformative prior or diffuse prior expresses vague or general information about a variable. The term "uninformative prior" is somewhat of a misnomer. Such a prior might also be called a not very informative prior, or an objective prior, i.e. one that's not subjectively elicited. Uninformative priors can express "objective" $\color{blue}{\text{information}}$ such as "the variable is positive" or "the variable is less than some limit".

My question is where is this $\color{blue}{\text{information}}$ given in my particular example, for instance?

What I've understand so far is that instead of saying "the variable is positive" or "the variable is less than some limit" in my case would be: the parameter $\theta$ has $x$ distribution, such distribution is given by the statistician.

Is it wrong?

Help please to understand this (If you could please don't introduce concepts of measure or measure sets because I don't have knowledge on this).

Note: Previously I asked this question (Information about parameters using priors distributions) and was marked as duplicate with other 5 links (What is an "uninformative prior"? Can we ever have one with truly no information?, Why would someone use a Bayesian approach with a 'noninformative' improper prior instead of the classical approach?, Why are Jeffreys priors considered noninformative?, What is the point of non-informative priors?, History of uninformative prior theory) I read them all and there was no answer to this...


Taking your example and adjusting it slightly to $\pi(\mu,\sigma^2)\propto\frac{1}{\sigma^2}$ similar to Wikipedia's example:

  • an argument that this prior is non-informative is that it is location-invariant and scale-invariant (uniform on the logarithmic scale), for example with properties that it leads to equal likelihoods for all possible values of the mean and that your results will be indifferent to the units of measurement (such as millimetres or kilometres);

  • an argument that this prior is informative is that it suggests that you think the mean is more likely to be greater in distance from $0$ than any particular large value you state, and that you think it more likely than not that the variance is either smaller than any particular small value you state or is greater than any large value you state; in other words it embodies the information that you believe it more likely than not that the mean and variance will be extreme to an incredible degree.

By the time you have some actual observations, these are less likely to be substantial arguments: with enough data, most moderately sensible priors produce broadly similar posterior distributions in most cases

| cite | improve this answer | |
  • $\begingroup$ (+1): adding a warning against over-interpretation of an improper prior as if it were a probability distribution, taking H. Jeffreys as an illustration of this over-interpretation. $\endgroup$ – Xi'an Feb 15 '19 at 11:26
  • $\begingroup$ What do you mean with this sentence, and that you think it more likely than not that the variance is ? I don't understand $\endgroup$ – Bellatrix Feb 15 '19 at 19:18
  • $\begingroup$ This part as well ...more likely than not that the mean... What did you mean? $\endgroup$ – Bellatrix Feb 15 '19 at 19:33
  • $\begingroup$ Choose any very very big number; this improper prior suggests you think in the absence of evidence that you are almost certain that the mean is bigger in magnitude than this. I wrote "more likely than not" as a more credible version of "almost certain" $\endgroup$ – Henry Feb 15 '19 at 20:14
  • 1
    $\begingroup$ I would think you need a course on Bayesian statistics to build this background. $\endgroup$ – Xi'an Feb 22 '19 at 21:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.