# Joint distribution of difference of normals

Consider the following three independent random variables, $$X_1 \sim N(\mu_1,\sigma^2_1)$$, $$X_2 \sim N(\mu_2,\sigma^2_2)$$, and $$X_3 \sim N(\mu_3,\sigma^2_3)$$.

Now let me define the pairwise differences as $$\delta_{12} = X_1-X_2$$, $$\delta_{13} = X_1-X_3$$, and $$\delta_{23} = X_2-X_3$$.

I know that these pairwise differences are easy to calculate as normal random variables, e.g.,

$$\delta_{ij}\sim N(\mu_i - \mu_j, \sigma^2_i + \sigma^2_j)$$

but are we able to calculate the joint distribution of $$\delta_{12},\delta_{13},\delta_{23}$$ in closed form?

In other words, what is the joint distribution of $$p(\delta_{12},\delta_{13},\delta_{23}| \mu_1, \mu_2,\mu_3,\sigma^2_1,\sigma_2^2,\sigma_3^2)$$?

Or do we need to place more assumptions?

Write \begin{align} \Delta := \begin{bmatrix} \delta_{12} \\ \delta_{13} \\ \delta_{23} \end{bmatrix} = \begin{bmatrix} X_1 - X_2 \\ X_1 - X_3 \\ X_2 - X_3 \end{bmatrix} = \begin{bmatrix} 1 & -1 & 0 \\ 1 & 0 & -1 \\ 0 & 1 & -1 \end{bmatrix} \begin{bmatrix} X_1 \\ X_2 \\ X_3 \end{bmatrix} =: AX, \end{align} then $$\Delta \sim N_3(AE(X), A\operatorname{Cov}(X)A^T)$$ by Affine transformation property of MVN. Given that \begin{align} E(X) = \begin{bmatrix} \mu_1 \\ \mu_2 \\ \mu_3 \end{bmatrix}, \quad \operatorname{Cov}(X) = \begin{bmatrix} \sigma_1^2 & 0 & 0 \\ 0 & \sigma_2^2 & 0 \\ 0 & 0 & \sigma_3^2 \end{bmatrix}, \end{align} can you finish the calculation?

• Indeed I can. Thank you so much Commented Nov 16, 2022 at 18:13
• Is this a degenerate normal distribution? Commented Dec 2, 2022 at 22:41
• @JohnSmith Yes, because $\operatorname{rank}(A) = 2 < 3$. Commented Dec 2, 2022 at 22:43
• Given that fact, is there a way to work around it so that I can still sample from the distribution? Or rather does that create an issue? Commented Dec 2, 2022 at 22:46
• It means that you don't need to consider the joint distribution of $\delta_{12}, \delta_{13}, \delta_{23}$. Instead, just consider the joint distribution of any two of the three. It's similar to in ANOVA, you set up a baseline level, and focus only on the difference between other levels and this fixed level. Commented Dec 2, 2022 at 22:52

It's trivial. The marginal means and variances remain the same, i.e. exactly as you have computed them. All you need to know is to calculate the off-diagonal of the variance-covariance matrix. To do so, this simple (useful) formula tells you what you need to know:

$$\text{var}(X_1 - X_2) = \text{var}(X_1) + \text{var}(X_2) - 2 \text{cov}(X_1, X_2)$$

• This only characterizes the mean vector and the covariance matrix but does not say what the joint distribution is. Commented Nov 16, 2022 at 17:20
• The OP has already stated that the differences are marginally normal, hence we need only characterize the covariance matrix to complete the problem. Commented Nov 16, 2022 at 17:53
• @AdamO OP eventually asked for joint distribution. Commented Nov 16, 2022 at 18:07
• @AdamO, marginal normality + mean vector + covariance matrix does not imply a joint distribution, or does it? Commented Nov 16, 2022 at 18:47
• @RichardHardy I probably learned my math facts in 1st year prob/stats in a different order than others, but I assumed the joint dist'ns of normals - under constraints already implied by OP - was already established. Commented Nov 16, 2022 at 19:48