5
$\begingroup$

Suppose $X$ and $Y$ have conditional distributions given by: \begin{align} f(x|y)&\propto ye^{-yx}\;\;\text{for}\;\;0<x<B<\infty\\ g(y|x)&\propto xe^{-xy}\;\;\text{for}\;\;0<y<B<\infty \end{align}

According to the article, the marginal distribution $g(x)$ is not easy to calculate, but it exists since $B<\infty$.

It is easy to determine that: \begin{align} f(x|y)&= \frac{e^B}{e^B-1}ye^{-yx}\;\;\text{for}\;\;0<x<B<\infty\\ g(y|x)&= \frac{e^B}{e^B-1}xe^{-xy}\;\;\text{for}\;\;0<y<B<\infty \end{align}

Now since $f(x|y)f(y)=f(x,y)=g(y|x)g(x)$, this means that: $$\frac{f(y)}{g(x)}=\frac{g(y|x)}{f(x|y)}=\frac{x}{y}.$$

And thus that: $$\frac{1}{g(x)}=\frac{1}{g(x)}\int_0^Bf(y)dy=\int_0^B\frac{f(y)}{g(x)}dy=\int_0^B\frac{x}{y}dy=x\ln(y)\Big|_{y=0}^{y=B}=-\infty.$$


This would imply that $g(x)=0$, and by a similar argument, that $f(y)=0$.

I'm not entirely sure what to make of this. Am I to conclude that since the ratio of marginals is an indeterminate form (zero over zero), that this method of calculating them fails in this particular case? and that they aren't in fact both zero.

Also, is this a common technique for calculating marginals given the conditionals? I came up with it while trying to see why the technique of Gibbs sampling couldn't be replaced by the (possibly numerical) evaluation of some (possibly quite difficult) integral.

Finally, like I mentioned above, the article says the marginals are not easy to calculate, which seems to imply that one could in theory calculate them, how would this be done?

$\endgroup$
7
  • 1
    $\begingroup$ You cannot write $\dfrac{f(x)}{g(x)}$, when $g(x)=0$. If you write $\dfrac{f(x)}{g(x)}$, then you are implicitly assuming that $g(x)>0$. $\endgroup$
    – Stat
    Nov 9, 2013 at 22:07
  • 3
    $\begingroup$ Perhaps I am going blind, but are you certain that your conditional densities integrate to unity over their domains and for an arbitrarily fixed value of the conditioning variable? $\endgroup$ Nov 10, 2013 at 0:29
  • $\begingroup$ @Alecos, ah!!!! there's my mistake, man I've been agonizing over this problem, thanks. $\endgroup$
    – Set
    Nov 10, 2013 at 0:32
  • 1
    $\begingroup$ As far as I can tell your discovery of my mistake solves the problem, the constant which doesn't depend on $y$ is $\frac{e^{Bx}}{e^{Bx}-1}$. The article is on JSTOR, "Explaining the Gibbs Sampler", by Casella and George $\endgroup$
    – Set
    Nov 10, 2013 at 0:51
  • 1
    $\begingroup$ The marginal density will then be $g(x)=\frac{e^{Bx}-1}{xe^{Bx}}$, which I determined just from fiddling around with $f(x|y)f(y)=g(y|x)g(x)$. $\endgroup$
    – Set
    Nov 10, 2013 at 0:53

1 Answer 1

3
$\begingroup$

This example is taken from the introductory Casella's and George's "Gibbs for kids" that was later retitled "Explaining the Gibbs sampler" as the American Statistician editor at that time objected to the original (and better) title...

The example is intended to demonstrate that well-defined and standard conditional distributions as those two, namely an $\text{Exp}(x)$ distribution for $f(y|x)$ and an $\text{Exp}(y)$ distribution for $f(x|y)$ may not correspond to any joint distribution $f(x,y)$. Hence, a Gibbs sampler based on those two conditionals will not converge to a stationary distribution but instead produces a null recurrent or transient Markov chain. The fact that the joint does not exist follows from Bayes' theorem: $$\frac{f(x|y)}{f(y|x)}=\frac{f(x)}{f(y)}=\frac{y}{x}$$since this implies that $f(x)\propto x^{-1}$ which is not integrable over $(0,\infty)$. This also implies that the invariant measure of the Markov chain has density $\exp\{-xy\}$ wrt the Lebesgue measure.

Now, if we move to the bounded case, by restricting the support to the square $(0,B)^2$, the full conditionals are $$f(x|y)=\frac{y\exp\{-xy\}}{1-\exp\{-By\}}\quad\text{and}\quad f(y|x)=\frac{x\exp\{-xy\}}{1-\exp\{-Bx\}}$$and not what you wrote in the question. This means that $$\frac{f(x|y)}{f(yx)}=\frac{f(x)}{f(y)}=\frac{y}{x}\times\frac{1-\exp\{-Bx\}}{1-\exp\{-By\}}$$So $$f(x)\propto\frac{1-\exp\{-Bx\}}{x}\quad\text{and}\quad f(y)\propto\frac{1-\exp\{-By\}}{y}$$which are integrable over $(0,B)$ since $$1-\exp\{-Bx\}\approx Bx-\frac{(Bx)^2}{2}$$ But the integral of the rhs of $f(x)$ is not simple, as it involves the exponential integral function $\text{Ei}(x)$ which is more easily expressed through a series expansion, $$\int_0^b \frac{1-\exp\{-x\}}{x}\text{d}x=- \sum_{k=1}^\infty \frac{(-b)^k}{k\,(k!)}$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.