# sampling from a multivariate normal distribution

I have a covariance matrix representing a multivariate normal distribution. I would like to draw samples from this distribution, e.g. using R's mvrnorm.

What happens if I diagonalize the covariance matrix in advance? This would yield a diagonal matrix, with the corrected variances on the diagonal. Can I now sample from each gaussian independently? if so, what is the new mean?

• If you replace the initial covariance with a diagonal one, this modifies the distribution you are sampling from. – Xi'an May 27 '18 at 8:41
• My space is very sparse because this is a very high-dimensional Gaussian. Hence, few samples don't represent the initial distribution well. Is there a way to improve the situation? – Omry Atia May 27 '18 at 8:45

## 1 Answer

Recall the formula $\text{Cov}(CX) = C \,\text{Cov}(X) \, C^T$ for how to compute the covariance matrix of a linear transform of a vector $X$, described by the matrix $C$. This shows that your method is indeed correct, however you need to linearly transform your independent Gaussians (put into a vector) to get samples from the original, correlated Gaussian vector. The mean will transform linearly; so in particular, if the original mean vector is zero, so is the one of the transformed vector.

The technique of principal components analysis is based on choosing not only a diagonalising $C$, but also on putting the eigenvectors (principal components in statistical language) belonging to the largest eigenvalues first. It may be best to use a routine in R that computes only the first few eigenvectors / principal components.