# Drawing from the joint distribution by drawing from the marginal and conditional distribution

How can we prove that a draw $$(x,y)$$ from the joint probability distribution of two random variables $$X$$ and $$Y$$ can be obtained by first generating a draw $$x$$ from the marginal probability distribution of $$X$$ and then a draw $$y$$ from the conditional probability distribution of $$Y$$ given $$X=x$$?
Is this a simple consequence of the definition of a conditional probability distribution?

In short, drawing from the joint distribution is equivalent to first drawing from one marginal and second from the corresponding conditional.

To answer a side question from @statmerkur in the comments, drawing from a distribution $$F$$ means producing a realisation of a random variable with distribution $$F$$, either by inducing the phenomenon distributed as $$F$$ (e.g., the time to an electron emission) or by simulating it (e.g., taking $$F^{-1}(U)$$ with $$U$$ a uniform variate when the distribution is univariate).

Considering that$$\mathbb P(X\in A,\ Y\in B)=\mathbb P(X\in A)\mathbb P(Y\in B|X\in A)$$ $$-$$where $$\mathbb P(\cdot)$$ denotes the probability of the event between parentheses$$-$$, we have \begin{align} \mathbb P\{X\in (\epsilon,\epsilon+\text d\epsilon),\ Y\in (\eta,\eta+\text d\eta)\}&=\int_{\epsilon}^{\epsilon+\text d\epsilon} \int_{\eta}^{\eta+\text d\eta} f_{Y|X}(y|x)\,\text dy\,f_X(x)\,\text dx\\ &=\mathbb E^X[\mathbb I_{(\epsilon,\epsilon+\text d\epsilon)}(X)\times\mathbb E^{Y|X}\{\mathbb I_{(\eta,\eta+\text d\eta)}(Y)|X\}] \end{align} where [in reply to @singlemalt]

1. $$f_{Y|X}(\cdot|\cdot)$$ denotes the probability density of the conditional distribution of $$Y$$ given $$X$$
2. $$\mathbb I_A(X)$$ is the indicator function that $$X\in A$$, equal to either 0 or 1, with $$\mathbb P(X\in A)=\mathbb E^X[\mathbb I_A(X)]$$
3. $$\mathbb E$$ is a blackboard-bold "E" that is often used for representing `expectations'
4. the superscript $$X$$ is indicating that the expectation is wrt the distribution of the rv $$X$$,
5. and the superscript $$Y|X$$ is indicating that the expectation is computed wrt the conditional distribution of the rv $$Y$$ given the rv $$X$$, meaning that $$X$$ is considered as fixed when computing this conditional expectation.

Therefore, if [one simulates] $$X\sim f_X(x)$$, with realisation $$x$$, and [then one simulates] $$Y\sim f_{Y|X}(y|x)$$, $$\mathbb I_{(\epsilon,\epsilon+\text d\epsilon)}(X)\times\mathbb I_{(\eta,\eta+\text d\eta)}(Y)$$ is an unbiased estimator of $$\mathbb E^{(X,Y)}[\mathbb I_{(\epsilon,\epsilon+\text d\epsilon)}(X)\times\mathbb I_{(\eta,\eta+\text d\eta)}(Y)]=\mathbb E^X[\mathbb I_{(\epsilon,\epsilon+\text d\epsilon)}(X)\times\mathbb E^{Y|X}\{\mathbb I_{(\eta,\eta+\text d\eta)}(Y)|X\}]$$ for all $$(\epsilon,\eta)$$. Letting $$\text d\epsilon$$ and $$\text d\eta$$ go to zero (0) allows one to conclude that the joint distribution of $$(X,Y)$$ is obtained through this decomposition marginal-then-conditional, thus that the simulation is producing a realisation from the correct (joint) distribution.

Take the example of a bivariate Normal $$(X,Y)^\prime\sim\mathcal N_2(0_2,\Sigma)\qquad \Sigma=\left(\begin{matrix}1 &\rho\cr\rho &1\end{matrix} \right)\qquad \rho\in(-1,1)$$ Then drawing $$X$$ from the marginal $$\mathcal N(0,1)$$ is equivalent to setting $$X=\epsilon_x$$ with $$\epsilon_x\sim \mathcal N(0,1)$$. Further, drawing $$Y$$ from the conditional $$\mathcal N(\rho x,1-\rho^2)$$ is equivalent to setting $$Y=\rho x +\sqrt{1-\rho^2}\epsilon_y\qquad\epsilon_y\sim\mathcal N(0,1)$$
Therefore $$\left(\begin{matrix}X\cr Y\cr\end{matrix} \right)= \overbrace{\left(\begin{matrix}1 &0\cr \rho &\sqrt{1-\rho^2}\cr\end{matrix} \right)}^A \left(\begin{matrix}\epsilon_x\cr \epsilon_y\cr\end{matrix} \right) \sim \mathcal N_2(0_2,\Sigma)$$ since $$A A^\mathsf{T}=\Sigma$$ This implies that the draw is truly from the joint distribution.

• Xi'an, what is the superscript $X$ on the exponential symbol? Does the thin hollow rectangle symbol represent identity? Aug 2, 2022 at 16:18