0
$\begingroup$

I have a covariance matrix representing a multivariate normal distribution. I would like to draw samples from this distribution, e.g. using R's mvrnorm.

What happens if I diagonalize the covariance matrix in advance? This would yield a diagonal matrix, with the corrected variances on the diagonal. Can I now sample from each gaussian independently? if so, what is the new mean?

$\endgroup$
  • 1
    $\begingroup$ If you replace the initial covariance with a diagonal one, this modifies the distribution you are sampling from. $\endgroup$ – Xi'an May 27 '18 at 8:41
  • $\begingroup$ My space is very sparse because this is a very high-dimensional Gaussian. Hence, few samples don't represent the initial distribution well. Is there a way to improve the situation? $\endgroup$ – Omry Atia May 27 '18 at 8:45
1
$\begingroup$

Recall the formula $\text{Cov}(CX) = C \,\text{Cov}(X) \, C^T$ for how to compute the covariance matrix of a linear transform of a vector $X$, described by the matrix $C$. This shows that your method is indeed correct, however you need to linearly transform your independent Gaussians (put into a vector) to get samples from the original, correlated Gaussian vector. The mean will transform linearly; so in particular, if the original mean vector is zero, so is the one of the transformed vector.

The technique of principal components analysis is based on choosing not only a diagonalising $C$, but also on putting the eigenvectors (principal components in statistical language) belonging to the largest eigenvalues first. It may be best to use a routine in R that computes only the first few eigenvectors / principal components.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.