4
$\begingroup$

Let $(X, Y)$ be a Gaussian random vector with mean $\mu$, variance $\Sigma$ and both $X$ and $Y$ are vectors. No structure is assumed on $\Sigma$ (aside from being symmetric positive definite).

Is there a way of calculating $(X - \mu_x)^T \Sigma_{xx}^{-1} (X - \mu_x)$ and getting the parameters of the conditional distribution of $Y|X$, $\mu_{Y|X} = \mu_y + \Sigma_{yx}\Sigma_{xx}^{-1}(X - \mu_x)$ and $\Sigma_{Y|X} = \Sigma_{yy} - \Sigma_{yx}\Sigma_{yy}^{-1}\Sigma_{xy}$ without having to invert $\Sigma$?

Asking because the "golden rule of numerical linear algebra" suggests that matrix inversion can usually be avoided if you are smart enough, but I am not very smart. Or is this a situation where you really do need to do a matrix inversion? In retrospect this seems more likely to be the case since people often impose structure on $\Sigma$ for computational reasons.

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ I suppose it depends on (a) what you end up wanting to do with these quantities and (b) what you mean by a "matrix inversion". Oftentimes you avoid matrix inversion by simply solving the right system of linear equations using a convenient decomposition. Here, worst case, you can use the SVD by writing $\Sigma_{xx}^{-1} = U D^{-1} U'$ where $\Sigma_{xx} = U D U'$, so only (explicit) inversion of a diagonal matrix is needed. $\endgroup$
    – cardinal
    Aug 25, 2012 at 3:32
  • $\begingroup$ I don't think matrix inversion can be avoided. Even in the univariate case, the conditional variance of $Y$ given $X$ is $$\sigma_Y^2 - \sigma_Y^2\rho^2 = \sigma_Y^2 - (\text{cov}(X,Y))^2/\sigma_X^2 = \sigma_Y^2 - \text{cov}(X,Y)\sigma_X^{-2}$$ that is, a division by $\sigma_X^2$ (or equivalently, multiplication by the multiplicative inverse $\sigma_X^{-2}$) is needed. $\endgroup$
    – Dilip Sarwate
    Aug 25, 2012 at 3:37

2 Answers 2

Reset to default
2
$\begingroup$

Say $b = \Sigma^{-1} (X- \mu_X)$ for some $b$, so that $(X- \mu_X)^T \Sigma^{-1} (X- \mu_X) = (X- \mu_X)^T b$.

This $b$ can be written as a linear equation by noticing that $\Sigma b = (X - \mu_X)$.

So, we can just solve for $b$, and then multiply $(X - \mu_X) b$.

Improve this answer
$\endgroup$
0
$\begingroup$

Form the Cholesky factor $R$ so that you have $\Sigma = R R^\top$. Then the quadratic form $x^\top \Sigma^{-1} x = z^\top z$ is computed by back-solving $R z = x$. Since Cholesky factors are triangular, solving this linear system is cheap and doesn't involve explicit inversion.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.