# do logs modify the correlation between two variables?

I am applying logs to two very skewed variables and then doing the correlation. Before logs the correlation is 0.49 and after logs it is 0.9. I thought the logs only change the scale. How is this possible? Here below the graphs for each of them. Perhaps I haven't applied the right transformation?

• Logarithms are manifestly a nonlinear transformation and so in general correlations will change, often substantially. You just found that out. It's not a problem; on the contrary, it is usually desired behaviour. The correlation is a measure of how far data can be approximated by a straight line; far from needing another transformation, you are evidently using the most appropriate transformation possible. Commented Dec 8, 2014 at 9:43
• Note that skewness here is evident and important, but secondary. The most important reason for transformation is that the relationship on the original scale is nonlinear. Commented Dec 8, 2014 at 9:44
• @NickCox: I'd quibble with "the correlation is a measure of how far data can be approximated by a straight line". This is correct for Pearson's correlation, but there are other correlation coefficients without that linearity assumption, and sometimes these are more suitable than finding a transformation and then applying Pearson's correlation. Commented Dec 8, 2014 at 9:46
• You're correct, naturally, and I would make the same correction in reverse. It's too late to edit my comment, but I meant "as used here". I think "correlation" does default to "Pearson correlation", but you are quite right to want explicit statements. (I first calculated Spearman correlations in 1965!) Commented Dec 8, 2014 at 9:50
• Taking logs can even change the sign of the Pearson correlation ... but the rank correlations won't change at all. Commented May 8, 2017 at 7:15

There are multiple different types of correlation. The most common one is Pearson's correlation coefficient, which measures the amount of linear dependence between two vectors. That is, it essentially lays a straight line through the scatterplot and calculates its slope. This will of course change if you take logs!

If you are interested in a measure of correlation that is invariant under monotone transformations like the logarithm, use Kendall's rank correlation or Spearman's rank correlation. These only work on ranks, which do not change under monotone transformations.

Here is an example - note how the Pearson correlation changes after logging, while the Kendall and the Spearman ones don't:

> set.seed(1)
> foo <- exp(rnorm(100))
> bar <- exp(rnorm(100))
>
> cor(foo,bar,method="pearson")
[1] -0.08337386
> cor(log(foo),log(bar),method="pearson")
[1] -0.0009943199
>
> cor(foo,bar,method="kendall")
[1] 0.02707071
> cor(log(foo),log(bar),method="kendall")
[1] 0.02707071
>
> cor(foo,bar,method="spearman")
[1] 0.03871587
> cor(log(foo),log(bar),method="spearman")
[1] 0.03871587


The following earlier question discusses Kendall's and Spearman's correlation: Kendall Tau or Spearman's rho?

• Spearman rank correlation is also possible here. Commented Dec 8, 2014 at 9:45
• And if I wanted to put together these two variables in a PCA, how would I do it? With the logs applied? Commented Dec 8, 2014 at 10:30
• And if I want to include these variables in a clustering procedure? What is it better? Can I say that I include only one because they are highly correlated? Do I need to include the logged variable (it is going to be more difficult to interpret it. Commented Dec 8, 2014 at 10:32
• At some point, it makes sense to ask a new question, because by now, we have left the correlation issue far behind. @NickCox wrote some very good comments. Looking at the scatterplot of your logged data, I'd recommend taking logs and including only one of the two variables, since they apparently carry pretty much the same information. Commented Dec 8, 2014 at 10:37
• The correlation coefficient does not "essentially lays a straight line through the scatterplot and calculates its slope". The correaltion is a measure of average deviation from that straight line.
– Omri
Commented Dec 14, 2018 at 7:20