Suppose that $X$ is a $k$ dimensional normal variate with diagonal covariance matrix. $$ X \sim N(\mu, \Sigma), $$ where $\Sigma=\textrm{diag}(\sigma_i^2)$. The problem I am trying to solve it to find the joint distribution for the difference between $X_i-X_1 \ \ \forall \ \ i>1$: $$ X_{2:k} - X_1 \sim \ ? $$

My progress so far:

The above problem can be framed as a hierarchical model, where $$ X_{2:k} - X_1 \ | \ X_1 \sim N(\mu_{2:k} - X_1, \textrm{diag}(\sigma_{2:k}^2)) $$ $$ X_1 \sim N(\mu_1,\sigma^2_1) $$ I've tried writing out the likelihood and integrating out $X_1$, but have so far been unable to get the arithmetic to work out. Finding the distribution of the individual components $X_i-X_1$ for $i>1$ is easy, as it is just the difference of independent normal variates $$ X_i - X_1 \sim N(\mu_i-\mu_1,\sigma_i^2+\sigma_1^2) $$

However, at this point I'm stuck. I'm pretty sure the desired joint distribution is normal, but I can't figure out the covariance between $X_i-X_1$ and $X_j-X_1$.

  • $\begingroup$ Please add the [self-study] tag & read its wiki. $\endgroup$ Commented Mar 31, 2016 at 19:10
  • $\begingroup$ Thank you for your comment. This is for my work, not for a course. If it is routine, then I have not seen it before, nor have I been able to find it with my friend google. A reference would be much appreciated! That said, since I'm not to familiar with the standards around here, if you still feel it is better under that tag I'll add it. $\endgroup$
    – rasta
    Commented Mar 31, 2016 at 19:25
  • $\begingroup$ My apologies, I thought you were introducing it as a homework problem that you were stuck on. You needn't add the tag. $\endgroup$ Commented Mar 31, 2016 at 19:27

1 Answer 1


Alrighty, I don't know why I wasn't able to see this before, but $$ cov(X_i-X_1,X_j-X_1) = E((X_i-X_1)(X_j-X_1)) - E(X_i-X_1)E(X_j-X_1) $$ Using the fact that $X_i$ and $X_j$ are independent for $j\neq i$, this reduces to $$ cov(X_i-X_1,X_j-X_1) = \sigma^2_1 \ \ \ \ for \ \ i \neq j $$ and $$ cov(X_i-X_1,X_j-X_1) = \sigma^2_i+\sigma^2_1 \ \ \ \ for \ \ i = j $$

So, we know the means and covariances. The distribution is Normal because the normal distribution is its own conjugate prior.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.