7
$\begingroup$

I am trying to learn the basics of Bayesian decision and I came across the phrase "proper prior" but I don't really understand what it means. Does anyone know?

$\endgroup$
  • 1
    $\begingroup$ By default "prior" commonly means "proper prior", so this is a case where looking for the antonym may be more useful. (If this does not help, you can clarify, but consider adding the self-study tag.) $\endgroup$ – GeoMatt22 Oct 8 '16 at 6:51
  • 1
    $\begingroup$ I searched for the meaning of improper prior and I found that it means the prior does not integrate to one and can even be infinite (it is not a proper probability distribution so to speak...) Do you agree@GeoMatt22? Also, what about the self-sudy tag? $\endgroup$ – Learn_and_Share Oct 8 '16 at 7:08
  • 1
    $\begingroup$ You are right, an improper prior is a prior that does not integrate to one and may even be infinite like e.g. a ''uniform prior'' over $[0,+\infty[$. $\endgroup$ – user83346 Oct 8 '16 at 7:29
  • $\begingroup$ @MedNait that is correct. A proper prior is literally a prior that is a PDF, so has unit integral. I mentioned self-study as your short question came across as similar to thse types of (homework/textbook) questions we see a lot, with little sense that the poster tried to solve on their own. But I realize that "proper prior" may be hard to Google, as noted in my original comment. $\endgroup$ – GeoMatt22 Oct 8 '16 at 8:20
7
$\begingroup$

A prior distribution that integrates to 1 is a proper prior, by contrast with an improper prior which doesn't.

For example, consider estimation of the mean, $\mu$ in a normal distribution. the following two prior distributions:

$\qquad f(\mu) = N(\mu_0,\tau^2)\,,\: -\infty<\mu<\infty$

$\qquad f(\mu) \propto c\,,\qquad\qquad -\infty<\mu<\infty.$

The first is a proper density. The second is not - no choice of $c$ can yield a density that integrates to $1$. Nevertheless, both lead to proper posterior distributions.

See the following posts which throw additional light on the use of improper priors issue and some closely related issues:

Flat, conjugate, and hyper- priors. What are they?

What is an "uninformative prior"? Can we ever have one with truly no information?

$\endgroup$
  • $\begingroup$ Just to complete my understanding, is a prior that integrates to 1 and is not always positive proper? $\endgroup$ – Learn_and_Share Oct 8 '16 at 11:46
  • $\begingroup$ That won't be any use at all, since it won't leave you with a proper posterior. $\endgroup$ – Glen_b Oct 8 '16 at 11:49
  • $\begingroup$ So basically, we can get a proper posterior even with an improper prior $p(\theta)$ because of the multiplication by the likelihood ($p(x|\theta)$: distribution of data given $\theta$ that cannot be improper!) that guaranties this. On the other hand, if the prior is negative valued nothing can correct for this. Is this correct? $\endgroup$ – Learn_and_Share Oct 8 '16 at 12:04
  • $\begingroup$ The product of a prior density that goes below with a likelihood will produce a posterior that goes below zero (unless the likelihood happens to be 0 there). But the prior going negative would also have no interpretation. An improper prior can generally still be understood in some (more or less vague) sense. $\endgroup$ – Glen_b Oct 8 '16 at 12:30
  • $\begingroup$ By: "An improper prior can still be understood in some sense" you mean as a weighting of the likelihood function, where the possibly high values of the improper prior can be corrected by the normalization factor $p(x)$? $\endgroup$ – Learn_and_Share Oct 8 '16 at 12:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.