# Derive the Gibbs sampler for this bivariate distribution

I understand the theory of Gibbs sampling. It is an iterative sampling algorithm that defines, sequence of random variables with the property of a Markov chain. Specifically, I choose any starting value, $(x_0, y_0)$ and discard the initial samples if my guess is poor (known as burn-in), then sample $x_1 \sim p(x|y_0)$ and $y_1 \sim p(y|x_1)$ for the first iteration. How do I derive the Gibbs sampler for the bivariate distribution with pdf:

$p(x,y) \propto \mathbb{1}(|x - y| < c) \mathbb{1}(x,y \in (0,1))$

In the examples I've seen online, like on page 3 of Casella's 1992 paper (http://biostat.jhsph.edu/~mmccall/articles/casella_1992.pdf), the conditional distributions can be written as a known distribution with varying parameters. In the bivariate pdf above, I don't see how the indicator functions can be interpreted as a known distribution.

Specifically, how do I solve for $p(x|y)$ and $p(y|x)$?

My work thus far:

$$\begin{array}{lcl} p(x,y) &\propto & p(x|y) \\[2ex] &\propto & \mathbb{1}(-c < |x-y| < c) \mathbb{1}(x,y \in (0,1)) \\[2ex] &\propto & \mathbb{1}(-c +y < x < c + y) \mathbb{1}(x \in (0,1)) \\[2ex] p(x,y) &\propto & p(y|x) \\[2ex] &\propto & \mathbb{1}(-c < |x-y| < c) \mathbb{1}(x,y \in (0,1)) \\[2ex] &\propto & \mathbb{1}(-c - x < -y < c - x) \mathbb{1}(x,y \in (0,1)) \\[2ex] &\propto & \mathbb{1}(c + x < y < -c + x) \mathbb{1}(y \in (0,1)) \\[2ex] \end{array}$$

• The Q&A format is not a suitable way to teach you Gibbs sampling; we need specific questions. Do you understand what Gibbs sampling consists of? Mar 3 '15 at 8:03
• Is the problem actually understanding the notation $1(|x,y| < c)$ rather than Gibbs sampling? Mar 3 '15 at 8:05
• @Glen_b Corrected my question. I am trying to solve for the conditional distribution $p(x|y)$ but am having trouble translating the indicator function into a known distribution. Mar 3 '15 at 12:57
• Note that Gibbs sampling is not about sequences of pairs; it works with no matter how many variables. Mar 3 '15 at 13:07
• @alberto You're right. Corrected. Mar 3 '15 at 13:19

First, it is useful to understand what $p(x,y)$ is. The superposition of an area which is $1\times\text{constant}$ (the proportionality constant that makes everything integrate to 1) in every point where $x$ and $y$ are at distance $d<c$ (if you go through the $x$ axis, you draw a band from $x+c$ to $x-c$) and a square from $(0,0)$ to $(1,1)$.

If your imagination fails, there is Wolfram Alpha :)

To get $p(x | y)$ you must get the slice of the figure taken at the given $y$. You see it is not the same slice at every $y$?. That means that, of course, $p(x | y)$ should include $y$ somewhere. Otherwise you are saying that $x$ is independent of $y$, that is $p(x | y) = p(x)$, and $p(x,y) = p(x)p(y)$, which would be a super boring joint distribution and you wouldn't need Gibbs sampling anymore.

Note: on "translating" the indicator function to a known distribution. What Gibbs sampling does is to attack every dimension separately. You don't know how to deal with the ugly bi-dimensional distribution of the figure. But you know how to deal with its uni-dimensional slices.

You just have to take the red pill and stop getting frightened by the bi-dimensional figure. We are doing Gibbs sampling, we live in 1-dimensional slices! ;)

• Thanks. This was helpful for my understanding, Alberto. However, I'm wondering if my derived mathematical statements for the conditionals are acceptable answers. Examples I've seen have always put the conditionals in terms of a known distribution. Mar 3 '15 at 17:05
• Your conditionals are wrong. The conditional on $y$ is a slice $[a,b]$ where $a$ and $b$ are defined with respect to $y$ (as the figure shows). Forget about probabilities and think geometrically. Then you'll see that the slice $[a,b]$ has the shape of a very, very, very trivial distribution. Mar 3 '15 at 17:18
• Ask yourself the following question: imagine $p(x,y)$ is the Pentagon building (though it would rather be the Hexagon). It has a flat roof. If you cut it with giant plane through some $y$, what would be the shape of that cut? This would be your distribution (up to a constant factor, of course, you just have to make it integrate to 1). Mar 3 '15 at 17:26