Marginal likelihood for simple hierarchical model

Question

Suppose that $X$ is a $k$ dimensional normal variate with diagonal covariance matrix. $$ X \sim N(\mu, \Sigma), $$ where $\Sigma=\textrm{diag}(\sigma_i^2)$. The problem I am trying to solve it to find the joint distribution for the difference between $X_i-X_1 \ \ \forall \ \ i>1$: $$ X_{2:k} - X_1 \sim \ ? $$

My progress so far:

The above problem can be framed as a hierarchical model, where $$ X_{2:k} - X_1 \ | \ X_1 \sim N(\mu_{2:k} - X_1, \textrm{diag}(\sigma_{2:k}^2)) $$ $$ X_1 \sim N(\mu_1,\sigma^2_1) $$ I've tried writing out the likelihood and integrating out $X_1$, but have so far been unable to get the arithmetic to work out. Finding the distribution of the individual components $X_i-X_1$ for $i>1$ is easy, as it is just the difference of independent normal variates $$ X_i - X_1 \sim N(\mu_i-\mu_1,\sigma_i^2+\sigma_1^2) $$

However, at this point I'm stuck. I'm pretty sure the desired joint distribution is normal, but I can't figure out the covariance between $X_i-X_1$ and $X_j-X_1$.

Answer 1

Alrighty, I don't know why I wasn't able to see this before, but $$ cov(X_i-X_1,X_j-X_1) = E((X_i-X_1)(X_j-X_1)) - E(X_i-X_1)E(X_j-X_1) $$ Using the fact that $X_i$ and $X_j$ are independent for $j\neq i$, this reduces to $$ cov(X_i-X_1,X_j-X_1) = \sigma^2_1 \ \ \ \ for \ \ i \neq j $$ and $$ cov(X_i-X_1,X_j-X_1) = \sigma^2_i+\sigma^2_1 \ \ \ \ for \ \ i = j $$

So, we know the means and covariances. The distribution is Normal because the normal distribution is its own conjugate prior.