Issues on computing Pearson correlation coefficient for two vectors

Question

When computing the Pearson correlation coefficient between a vector $X = \{x_1,...,x_n\}$ and a vector $Y = \{y_1,...,y_n\}$, we need to compute it as

$$\frac{1}{n}\sum_{i=1}^{i=n}(x_i-\bar{x})(y_i-\bar{y}).$$

Here $\bar{x} = (1/n)*\sum_{i=1}^{i=n}x_i$.

My question concerns the case where $x_i$'s, $i=1\dots,n$, are in different order of magnitude. For instance, when $x_1 = 1000$, and $x_2 = 1$ is there any specific procedure to preprocess these different $x_i$'s? Or can we directly compute $\bar{x}$ as the above? Is there any article or book discussing this topic?

Answer 1

Hi this should not be a problem since the mean is explicitly subtracted. Here's a small example (all codes in r):

require(mnormt)
#We create a multivariate Normal random variable
df<-rmnorm(n = 100, mean = rep(0, 2), matrix(c(1,0.5,0.5,1),nrow=2)) 

#We compute the correlation
cor(df)
        [,1]      [,2]
 [1,] 1.0000000 0.5605498
 [2,] 0.5605498 1.0000000

#We scale the first variable by 1000
df[,1] <- df[,1]*10000

#The correlation stays the same
cor(df)
         [,1]      [,2]
 [1,] 1.0000000 0.5605498
 [2,] 0.5605498 1.0000000

Hope this helps.

Edit Follow up to the comments (thanks to whuber): I did understand the question as being related to the magnitude of the whole vector. I understand from the discussion that some understood the question as being related to outliers. In this case my solution is, of course, not helpful.

Answer 2

This can be seen as a problem of outliers. As has been argued elsewhere (e.g. here or there), it is not a good idea to preprocess outliers automatically or to delete them blindly.

If you have outliers, you could use a Spearman correlation rather than a Pearson correlation. Spearman's coefficient computes the correlation between ranks, rather than between values. In this case, the gap between you first and your second observation does not matter.

Here is some R code to illustrate the point. If the last value of the vector x is rescaled by a factor 1000, you will see that Pearson's coefficient changes a lot, whereas Spearman's coefficient remains unchanged.

x <- seq(0:9)
y <- rnorm(10)
x
y

cor(x, y, method = "pearson")
cor(x, y, method = "spearman")

x[10] <- 1000
x
y

cor(x, y, method = "pearson")
cor(x, y, method = "spearman")

Answer 3

There are two aspects to this question.

1. The different orders of magnitude raise numerical problems. The naive way of computing them using a large sum-of-squares causes a loss in precision. This already happens for the means, but for the squares it is much worse. You can find some hints of solving this in the classic Knuth books.

2. Pearson correlation is optimal for estimating a linear trend. When you have such large differences in your magnitudes, you probably do not have linear data. Spearman correlation (pearson on ranks) may be more appropriate then, or e.g. taking the data to log-log-space first.

Answer 4

Divide through by the standard deviation of x and y to get Pearsons Correlation coefficicent and it should account for difference in magnitude.