2
$\begingroup$

Preface

I must say I am aware of previous discussions (e.g. this one) and also of this excellent, didactic proof using Fubini's theorem as presented by Jared Niemi [I'm not saying Jared Niemi is the originator of said proof, mind you]. But I have been thinking about an example where we have a likelihood with variable support and a proper prior and the posterior is improper. I do not claim to have discovered anything new; I'm most likely missing something obvious and would be grateful if someone could point out to me where my "counter-example" below fails.

The supposed counter-example

This is supposed to be a counter-example to the assertion "if the prior $\pi(\theta)$ is proper then the posterior $p(\theta \mid y)$ will be proper almost surely".

Let $p(y | \theta) = \frac{2y}{\theta}, \theta > 0, y \in [0, \sqrt{\theta}]$. It seems to me this likelihood is a proper distribution over $y$: it's non-negative and integrates to unity over the support. The slight quirk is that the support depends on the parameter. This is the case for instance for the GEV too. Now suppose I place a standard half-Cauchy prior on $\theta$, $ \pi(\theta) = 2/\pi(1 + \theta^2), \theta > 0$. It seems to me that the posterior $p(\theta \mid y)$ is not integrable w.r.t. $\theta$: $$ \int_0^{+\infty} p(\theta \mid y) d\theta = \int_0^{+\infty} \frac{2y}{\theta} \frac{2}{\pi \left( 1 + \theta^2 \right)} d\theta = \frac{4y}{\pi} \int_0^{+\infty} \frac{1}{\theta(1+\theta)^2} d\theta = \infty.$$

If someone could point out what I'm missing here, I'd be grateful.

What is wrong

The limits of integration (see Robin's answer).

$\endgroup$
4
$\begingroup$

The likelihood you are using is actually $$p(y|\theta)=\frac{2y}{\theta}\mathbb I_{\{0\leq y \leq \sqrt \theta\}}.$$ You didn't include the indicator in your posterior calculation. The actual posterior is defined by $$p(\theta|y)\propto \frac{2}{\pi(1+\theta^2)}\frac{2y}{\theta}\mathbb I_{\{\theta\geq y^2\}}.$$

The lower limit on your integral is thus wrong: the integral of the posterior is actually $$\int_{y^2}^\infty \frac{2}{\pi(1+\theta^2)}\frac{2y}{\theta} \,d\theta = \frac{4y}{\pi}\int_{y^2}^\infty \frac{1}{\theta(1+\theta^2)}\,d\theta = \frac{2y}{\pi}\left(\log(y^4+1)-4\log y\right)$$ which is finite as long as $y>0$, hence the posterior is proper almost surely.

$\endgroup$
  • $\begingroup$ There is no possibility that a proper prior leads to an improper posterior. $\endgroup$ – Xi'an Oct 8 '18 at 13:38
  • $\begingroup$ I know, @Xi'an. It's just that I was confused about what I was doing wrong in this particular example. The proof I linked, which uses Fubini's theorem, assumes the likelihood to be proper. This is almost always the case, but I wonder how you go about proving propriety when the likelihood is no longer integrable over the data space (e.g. the so-called "power priors"). $\endgroup$ – Luiz Max Carvalho Oct 8 '18 at 17:19
  • 1
    $\begingroup$ Using powered likelihoods is open to this danger of the posterior not being defined, simply because some forms of powered likelihoods are no longer integrable in $x$, the data. This means case-by-case examination must be conducted to ensure the posterior doth exist. $\endgroup$ – Xi'an Oct 8 '18 at 18:58
  • $\begingroup$ @Xi'an Do you have a canonical example of a powered likelihood and proper prior leading to an improper posterior? $\endgroup$ – Robin Ryder Oct 9 '18 at 7:44
  • 2
    $\begingroup$ @Xi'an I've created a question about this: stats.stackexchange.com/questions/370919/… $\endgroup$ – Robin Ryder Oct 9 '18 at 8:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.