Figuring out the bounds of integration over a joint pdf

Question

I have two random variables $X$ and $Y$ where the support of $X$ and $Y$ are the following: $0\leq X\leq 1$ and $0\leq Y\leq 1$. I also have their joint distribution, i.e., $f_{X,Y}(x,y)$ and I want to calculate the distribution of $X-Y$.

Now, I am doing this by a transformation of variables where $Z=X-Y$ and $W=X$ and I know that I need to calculate (which I can) the joint distribution of $Z$ and $W$, i.e., $f_{Z,W}(z,w)$.

Once I have that joint distribution, I can marginalize over $W$ to get the distribution that I ultimately want. However, what I am having trouble with is figuring out the bounds of integration for this integral, i.e., I have $$f_{X-Y}(x-y) = f_Z(z)=\int f_{Z,W}(z,w)dW$$ but the bounds of the integral elude me.

I also know the following is true

$$0\leq X\leq1\implies0 \leq W\leq1$$ and $$-1\leq X-Y\leq1\implies -1 \leq Z\leq1$$

So do I just need to integrate over 0 to 1 to integrate out $W$? Or am I missing something? A picture would be very helpful as well.

Answer 1

You can throw some values around for X and Y and see how the plot looks like. Usually, I use the endpoints and mid-points (sometimes these are useful, sometimes they are not) for both X and Y.

So, for instance: $$\begin{matrix} X & Y & X-Y \\ 0 & 0& 0 \\ 1 & 0 & 1\\ 0 & 1 & -1\\ 1 & 1 & 0\\ 0.5 & 1 & -0.5\\ 0.5 & 0 & 0.5\\ 0 & 0.5 & -0.5 \\ 1 & 0.5 & 0.5 \end{matrix}$$

And we can plot the values for $Z=X-Y$ and $W=X$:

Since we are interested in the bounds of integration for $W$, we interpret the plot above in terms of what happens to $W$ as $Z$ moves up and down. So, as $Z$ moves from $0$ to $-1$, $W$ leaves from $0$, passes through all the points in the blue area ($W < Z +1$) and hits the boundary $W=Z+1$. So, $\mbox{for } -1 \leq Z \leq 0, \, 0 \leq W \leq Z+1$.

Now, since we considered $0$ in our first interval, we can't consider it again when assessing the movement of $Z$ as it goes from $0$ to $1$. So, starting with $Z$ slightly above $0$ we see that $W$ falls into the grey area (i.e. $Z<W$) and moves upward until it reaches $W=Z$, and since the maximum value of $Z$ is $1$, we have that $\mbox{ for } 0 < Z \leq 1, \, Z \leq W \leq 1$.

Therefore $$ \begin{align} f_{X-Y}(x-y) = f_Z(z)= \begin{cases} \int_{0}^{z+1} f_{Z,W}(z,w)dW \, , &\mbox{if } -1 \leq z \leq 0 \\ \\ \int_{z}^{1} f_{Z,W}(z,w)dW \, , &\mbox{if } \phantom{-}0 < z \leq 1 \\ \\ 0 \, , &\mbox{otherwise} \end{cases} \end{align} $$

Answer 2

If you wanted, you could integrate over the entire set (e.g. $-\infty$ to $\infty$). Of course, this isn't necessary because $f_{Z,W}(z, w) = 0$ for most of these values, so the interval over which you integrate can be tightened. The key point is that, when $f_{Z,W}(z, w)$ is zero, it doesn't contribute anything to the integral. So, any values of $W$ for which $f_{Z,W}(z, w) = 0$ can be excluded from the interval.

You say that $W = X$ and the support of $p(X)$ is $[0, 1]$, so the support of $p(W)$ is also $[0, 1]$. This means that the marginal distribution $p(W)$ is zero for values outside this interval. Therefore, the joint distribution $f_{Z,W}(z, w)$ must also be zero when $W$ falls outside this interval. If this weren't the case, then integrating the joint distribution over $Z$ to obtain the marginal distribution $p(W)$ would give nonzero values for $W$ outside $[0, 1]$, in contradiction to the stated marginal distribution.

Now, combine the two statements. 1) You only need to integrate over the interval where $f_{Z,W}(z, w)$ is nonzero. 2) $f_{Z,W}(z, w) = 0$ when $W$ falls outside the interval $[0, 1]$. Therefore, you can integrate from $0$ to $1$.

You can check that this is true by integrating over larger intervals and seeing that the result doesn't change.

Here are some examples for different joint distributions of $X$ and $Y$.

Example 1: $X$ and $Y$ independent, both drawn from $U(0, 1)$

Example 2: $X \sim U(0, 1)$, $Y = X^2$

The joint distributions are visualized by sampling points, then using a 2d histogram. The marginal distribution $P(Z)$ is obtained by integrating/collapsing over $W$. Both cases show that the joint distribution of $Z$ and $W$ is zero when $W$ falls outside $[0, 1]$, so it's only necessary to integrate over $W \in [0, 1]$.

These examples also show that integrating over $W \in [0, 1]$ may not give the tightest possible integration bounds. In both examples, this interval includes regions where the joint distribution has value $0$. This will still produce the correct answer, because the zero values don't contribute to the integral. To obtain a tighter interval, you would need to take into account the dependence between $W$ and $Z$, which depends in turn on the joint distribution of $X$ and $Y$. In example 1, the joint distribution of $W$ and $Z$ is a parallelogram, so you could imagine varying the bounds of the integral as $Z$ changes. This would look like sliding a vertical 'window' horizontally over the parallelogram such that the window only contains points where the joint distribution is nonzero. The integral would be taken over the window at each position. But, in example 2, it's clear that the strategy would look different (you couldn't use a single interval).

So, integrating over $W \in [0, 1]$ is a simple strategy that should work in all cases. But if you want tight integration bounds, you'll have to use information about the dependence between $W$ and $Z$ to adjust the integration region as $Z$ changes.