Avoiding matrix inversions with the multivariate Gaussian distribution?

Question

Let $(X, Y)$ be a Gaussian random vector with mean $\mu$, variance $\Sigma$ and both $X$ and $Y$ are vectors. No structure is assumed on $\Sigma$ (aside from being symmetric positive definite).

Is there a way of calculating $(X - \mu_x)^T \Sigma_{xx}^{-1} (X - \mu_x)$ and getting the parameters of the conditional distribution of $Y|X$, $\mu_{Y|X} = \mu_y + \Sigma_{yx}\Sigma_{xx}^{-1}(X - \mu_x)$ and $\Sigma_{Y|X} = \Sigma_{yy} - \Sigma_{yx}\Sigma_{yy}^{-1}\Sigma_{xy}$ without having to invert $\Sigma$?

Asking because the "golden rule of numerical linear algebra" suggests that matrix inversion can usually be avoided if you are smart enough, but I am not very smart. Or is this a situation where you really do need to do a matrix inversion? In retrospect this seems more likely to be the case since people often impose structure on $\Sigma$ for computational reasons.

Answer 1

Say $b = \Sigma^{-1} (X- \mu_X)$ for some $b$, so that $(X- \mu_X)^T \Sigma^{-1} (X- \mu_X) = (X- \mu_X)^T b$.

This $b$ can be written as a linear equation by noticing that $\Sigma b = (X - \mu_X)$.

So, we can just solve for $b$, and then multiply $(X - \mu_X) b$.