3
$\begingroup$

I am a bit confused about improper priors and posteriors.

I have seen references that classify a prior or posterior probability density function as "improper" if the integral over infinite support sums to infinity. As far as being "proper", I have seen references which talk about if the integral is finite, i.e., $< \infty$, then it is proper.

I was under the impression we needed to integrate to 1.

$\endgroup$
8
$\begingroup$

Strictly speaking, an everywhere non-negative prior that integrated to some finite positive value other than 1 would not be a proper density and so arguably could in that sense be referred to as "improper", but since

(a) if it integrates to $k$, say, it's easy enough to scale if you need that

(b) frequently in Bayesian work we're only dealing with forms up to normalizing constants anyway, such as: (i) because of the need to integrate the denominator in Bayes' theorem so we'd just be applying a different scaling constant to that calculation (e.g. if we recognize the form of the posterior we can work out the right normalizing constant directly), or (ii) the need for explicit normalizing is removed by other considerations (such as if we're generating via accept-reject, say, we don't necessarily need the explicit normalizing constant)

So a prior for which we failed to give the normalizing constant to make it integrate to 1 doesn't usually pose any problem since we can get the result as if we had done so.

As a result, generally when people say "improper" they literally mean one with a non-finite integral, for which no "correct" normalizing constant exists

$\endgroup$
  • $\begingroup$ Thank you! When you refer to a normalizing constant, are you talking about the normalizing constant for the prior or is it in reference to the integral in the denominator of a posterior distribution? $\endgroup$ – user1398057 May 8 '16 at 2:12
  • $\begingroup$ It varies with context there; e.g. where I'm talking about a posterior (as in the part that mentioned accept-reject) then it's the normalizing constant for that (but that includes the normalizing constant for the prior in it). $\endgroup$ – Glen_b May 8 '16 at 6:02
6
$\begingroup$

A density function integrates to infinity, then the density function is termed as being "improper". A pdf is termed as being "proper" if it integrates to a finite quantity.

Ideally, a proper function will integrate to 1. However, sometimes density functions are known only up to a normalizing constant. That is $p(x) = c f(x)$, where $p(x)$ integrates to 1, but $c$ is unknown. In such a case, we might let $f(x)$ represents a "proper" density function, where it is understood that when we refer to $f(x)$ as a pdf, we in fact are referring to $p(x)$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.