3
$\begingroup$

When computing the Pearson correlation coefficient between a vector $X = \{x_1,...,x_n\}$ and a vector $Y = \{y_1,...,y_n\}$, we need to compute it as

$$\frac{1}{n}\sum_{i=1}^{i=n}(x_i-\bar{x})(y_i-\bar{y}).$$

Here $\bar{x} = (1/n)*\sum_{i=1}^{i=n}x_i$.

My question concerns the case where $x_i$'s, $i=1\dots,n$, are in different order of magnitude. For instance, when $x_1 = 1000$, and $x_2 = 1$ is there any specific procedure to preprocess these different $x_i$'s? Or can we directly compute $\bar{x}$ as the above? Is there any article or book discussing this topic?

$\endgroup$
  • 2
    $\begingroup$ That is not a correlation coefficient but $n$ times the covariance. Wikipedia gives stable one-pass and two-pass algorithms for calculating the covariance. $\endgroup$ – Henry Dec 21 '11 at 10:41
  • 2
    $\begingroup$ Could you elaborate on the nature of your concern? There does not seem to be any problem here. The formulas make sense for all $n\ge 1$ and can be computed no matter what values the $x_i$ take on. $\endgroup$ – whuber Dec 21 '11 at 16:45
2
$\begingroup$

Hi this should not be a problem since the mean is explicitly subtracted. Here's a small example (all codes in r):

require(mnormt)
#We create a multivariate Normal random variable
df<-rmnorm(n = 100, mean = rep(0, 2), matrix(c(1,0.5,0.5,1),nrow=2)) 

#We compute the correlation
cor(df)
        [,1]      [,2]
 [1,] 1.0000000 0.5605498
 [2,] 0.5605498 1.0000000

#We scale the first variable by 1000
df[,1] <- df[,1]*10000

#The correlation stays the same
cor(df)
         [,1]      [,2]
 [1,] 1.0000000 0.5605498
 [2,] 0.5605498 1.0000000

Hope this helps.

Edit Follow up to the comments (thanks to whuber): I did understand the question as being related to the magnitude of the whole vector. I understand from the discussion that some understood the question as being related to outliers. In this case my solution is, of course, not helpful.

$\endgroup$
  • 1
    $\begingroup$ You should add library(QRMlib) at the beginning of the code, rmnorm is not a base function. $\endgroup$ – Elvis Dec 21 '11 at 8:25
  • $\begingroup$ thanks a lot - i forgot. but i got this from require(mnormt). $\endgroup$ – Seb Dec 21 '11 at 8:27
  • $\begingroup$ I think the question was about rescaling the first observation of x, rather than rescaling the whole vector. $\endgroup$ – user5644 Dec 21 '11 at 8:59
  • $\begingroup$ well, i did read it differently since he states "$x_i$'s, $i=1\dots,n$"... anyway the author will chose his solution anyway. $\endgroup$ – Seb Dec 21 '11 at 9:07
  • 1
    $\begingroup$ Clarification is preferable, Seb. I did not mean to suggest your answer is wrong or inferior, but only that to be generally useful it ought to explain the sense in which you have interpreted the original question. $\endgroup$ – whuber Dec 22 '11 at 13:22
2
$\begingroup$

This can be seen as a problem of outliers. As has been argued elsewhere (e.g. here or there), it is not a good idea to preprocess outliers automatically or to delete them blindly.

If you have outliers, you could use a Spearman correlation rather than a Pearson correlation. Spearman's coefficient computes the correlation between ranks, rather than between values. In this case, the gap between you first and your second observation does not matter.

Here is some R code to illustrate the point. If the last value of the vector x is rescaled by a factor 1000, you will see that Pearson's coefficient changes a lot, whereas Spearman's coefficient remains unchanged.

x <- seq(0:9)
y <- rnorm(10)
x
y

cor(x, y, method = "pearson")
cor(x, y, method = "spearman")

x[10] <- 1000
x
y

cor(x, y, method = "pearson")
cor(x, y, method = "spearman")
$\endgroup$
1
$\begingroup$

There are two aspects to this question.

  1. The different orders of magnitude raise numerical problems. The naive way of computing them using a large sum-of-squares causes a loss in precision. This already happens for the means, but for the squares it is much worse. You can find some hints of solving this in the classic Knuth books.

  2. Pearson correlation is optimal for estimating a linear trend. When you have such large differences in your magnitudes, you probably do not have linear data. Spearman correlation (pearson on ranks) may be more appropriate then, or e.g. taking the data to log-log-space first.

$\endgroup$
-1
$\begingroup$

Divide through by the standard deviation of x and y to get Pearsons Correlation coefficicent and it should account for difference in magnitude.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.