Considering that you have to compute (large) correlation matrix between variables x, y, z,...etc, via matrix multiplications, will you prefer $\frac{cov_{xy}}{\sqrt{\sigma_x^2 \sigma_y^2}}$ or $\frac{cov_{xy}}{\sqrt{\sigma_x^2}\sqrt{\sigma_y^2}}$?

Note that the second is faster because you will take square roots from vector's elements, not from square matrix' elements. But there maybe problems with accuracy.

What are your thoughts, pros and cons?

Later update. (Thank @Martijn you brought me back to this old question)

Formula 2 often leaves diagonal entries of the matrix not all exactly $1$; the discrepancy is, of course, in the 15-16th decimal digit (i.e. the limit of the standard convention floating point computations). That makes it necessary to set $1$s on the diagonal after you computed the matrix by formula 2. Formula 1 does not has this problem.

However, setting a diagonal is usually a very fast operation. Will (A) using this operation with formula 2 be eventually faster or slower than (B) using formula 1 which is potentially a slower computation because it takes sq. root from more entries than formula 2 does?

My probes in SPSS (using matrix algebra syntax) demonstrated that A is just a wee bit faster than B; both computing large correlation matrices (5000x5000) or computing small correlation matrices repeatedly 50x50, 5000 times left B to fall 0.1 sec behind, overall, on my current system.

  • $\begingroup$ Are you talking about computational efficiency? $\endgroup$ – smndpln Dec 3 '16 at 12:21
  • $\begingroup$ Well, yes. And wider: speed/accuracy trade-off, results in near-singular data, etc. I'd like opinions on "best practice" wrt the question. I don't require specifically a formalized answer from a numerical geek, though will gladly welcome one. Thank you. $\endgroup$ – ttnphns Dec 3 '16 at 12:41
  • $\begingroup$ I do not see why there would be problems with accuracy. The square root is computed with very high precision so the increase in the size of the error due to the use of the second method will be negligible. $\endgroup$ – Sextus Empiricus Jan 21 '19 at 13:44
  • $\begingroup$ @MartijnWeterings, thank you. I finally (when have to program r calculation in SPSS myself) stack to the first formula. The second one sometimes leaves (in SPSS at least) diagonal entries of a corr matrix not exactly 1, so one has set 1 afterwards, which takes milliseconds. $\endgroup$ – ttnphns Jan 21 '19 at 17:25
  • $\begingroup$ That may depend on the way of computation. In R I get no discrepancies (using c(1:200)/(c(1:200)^2)^{1/2}-1) when I use the formula $$ x/(x^a)^{1/a}$$ in the case of $a=2$. Or, in fact, neither for other exponents of 2. But for $a \neq 2^k$ I indeed get small differences (small differences which I only get displayed when I view the numbers in high precision, that is why I subtract 1 from the ratio). Are these differences small in SPSS as well? I have errors of the order $10^{-16}$ which is the limit of the precision in the calculations with standard floating point variables. $\endgroup$ – Sextus Empiricus Jan 21 '19 at 17:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.