# Avoiding matrix inversions with the multivariate Gaussian distribution?

Let $(X, Y)$ be a Gaussian random vector with mean $\mu$, variance $\Sigma$ and both $X$ and $Y$ are vectors. No structure is assumed on $\Sigma$ (aside from being symmetric positive definite).

Is there a way of calculating $(X - \mu_x)^T \Sigma_{xx}^{-1} (X - \mu_x)$ and getting the parameters of the conditional distribution of $Y|X$, $\mu_{Y|X} = \mu_y + \Sigma_{yx}\Sigma_{xx}^{-1}(X - \mu_x)$ and $\Sigma_{Y|X} = \Sigma_{yy} - \Sigma_{yx}\Sigma_{yy}^{-1}\Sigma_{xy}$ without having to invert $\Sigma$?

Asking because the "golden rule of numerical linear algebra" suggests that matrix inversion can usually be avoided if you are smart enough, but I am not very smart. Or is this a situation where you really do need to do a matrix inversion? In retrospect this seems more likely to be the case since people often impose structure on $\Sigma$ for computational reasons.

-
I suppose it depends on (a) what you end up wanting to do with these quantities and (b) what you mean by a "matrix inversion". Oftentimes you avoid matrix inversion by simply solving the right system of linear equations using a convenient decomposition. Here, worst case, you can use the SVD by writing $\Sigma_{xx}^{-1} = U D^{-1} U'$ where $\Sigma_{xx} = U D U'$, so only (explicit) inversion of a diagonal matrix is needed. – cardinal Aug 25 '12 at 3:32
I don't think matrix inversion can be avoided. Even in the univariate case, the conditional variance of $Y$ given $X$ is $$\sigma_Y^2 - \sigma_Y^2\rho^2 = \sigma_Y^2 - (\text{cov}(X,Y))^2/\sigma_X^2 = \sigma_Y^2 - \text{cov}(X,Y)\sigma_X^{-2}$$ that is, a division by $\sigma_X^2$ (or equivalently, multiplication by the multiplicative inverse $\sigma_X^{-2}$) is needed. – Dilip Sarwate Aug 25 '12 at 3:37

Say $b = \Sigma^{-1} (X- \mu_X)$ for some $b$, so that $(X- \mu_X)^T \Sigma^{-1} (X- \mu_X) = (X- \mu_X)^T b$.
This $b$ can be written as a linear equation by noticing that $\Sigma b = (X - \mu_X)$.
So, we can just solve for $b$, and then multiply $(X - \mu_X) b$.