Is the correlation coefficient better than we think?

In order to measure precisely the influence of one variable $Y$ over $X$ I wanted to use the mutual information because so far I believed that correlation coefficient (Pearson) was only limited to linear relation.

But I realized that even in non-linear cases it gives good results.

Example in Matlab:

$X$ is a uniform random variable and $I$ is the range of $X$

$Y=X^{2},I=[0,100]: \rho=0.96$

$Y=\sqrt{X} ,I=[0,100]: \rho=0.94$

$Y=exp(X),I=[0,10]: \rho=0.70$

$Y=log(X),I=[0,100]: \rho=0.84$

Why the correlation is quite high in all those non-linear cases? I am beginning to think that correlation is only weak when the monotony of $X$ and $Y$ is not the same. Otherwise it is not so bad.

Do some people have a point of view on this?

• What is the meaning of $I=[0,100]$? For which values of $X$ did you calculate the correlation coefficient? – Alecos Papadopoulos May 22 '14 at 10:04
• "when the mononty is changing": I imagine that you want to make some point about monotonicity, or the lack of it, but I didn't feel confident at identifying exactly what you mean, so I left that unedited. Regardless of what you meant. it is easy to manufacture, or indeed find, cases with nonlinear and non-monotonic relationships where correlation is exactly or practically zero. – Nick Cox May 22 '14 at 11:04
• I is the range where $X$ takes its values. @NickCox I mean I feel like when the monotony of $X$ is following the monotony of $Y$ we often find a high correlation coefficient regardless the type of relation (linear or non linear) – Gandhi91 May 22 '14 at 11:32
• Sorry, but what you mean is completely unclear to me. It's the relation between the variables we are looking at. See e.g. en.wikipedia.org/wiki/Monotonic_function What precisely do you mean by the "monotony" of an individual variable? – Nick Cox May 22 '14 at 12:04
• "monotony" is a standard English word but it does not, so far as I know, have the mathematical meaning you seem to think it has. Monotony changing is just a matter of not being monotonic, or of being non-monotonic. I don't think this question is getting clearer or more helpful to anyone else. If the point is that monotonic nonlinear relationships can be associated with high (Pearson) correlation, a short answer is Indeed; and a slightly longer answer is that your teachers or textbooks were at fault if they didn't emphasise that. – Nick Cox May 22 '14 at 12:15

A couple of monotonic examples in R (but very similar code works in Matlab):

> x=(1:100)
> y=exp(x)
> z=log(x)
> cor(z,y,method="kendall")
[1] 1
> cor(z,y,method="spearman")
[1] 1
> cor(z,y)
[1] 0.1549211


That's pretty low!

It's quite possible to make it much lower:

> cor(xxx,yyy,method="spearman")
[1] 1
> cor(xxx,yyy,method="kendall")
[1] 1
> cor(xxx,yyy)
[1] 0.010647


and in fact that very low Pearson correlation is caused by only a single outlier in x and a single outlier in y:

> cor(xxx[2:99],yyy[2:99])
[1] 1


So even when 98% of the points lie on a perfectly straight line, and the relationship is perfectly monotonic, the Pearson correlation can still be really close to zero.

In the case where $x$ is restricted to take the values 1, 2, ..., 100, the correlation can still be very low. Here's an example:

 cor(1:100,exp(1:100))
[1] 0.252032


I'm not sure how you got 0.7 - perhaps I misunderstood. I have another example with equispaced $x$ that is about 0.175.

• I'm not sure to understand. In your first code you compute $corr(exp(X),log(X))$ I agree with the low coefficient. But it's not the example I was taking. In your last examples, I don't know what is 'xxx' and correlation is sensible to outliers, I agree. But my question was: why in the example of non linear cases I gave (without outliers) Pearson correlation is high? – Gandhi91 May 22 '14 at 11:47
• If you limit $x$ to be equispaced (was that the intent?), then the minimum correlation is higher (but still a good deal lower than in your examples; I've included one at about 0.25. but it's possible to get quite a bit smaller than that). I didn't tell you the content of the second set of variables deliberately but only a general sense of how to obtain them - sometimes it's fun to discover on ones own. – Glen_b May 22 '14 at 11:52

The more a function deviates from linearity, the smaller the correlation coefficient is.

Try this one:

> cor(1:100,factorial(1:100))
[1] 0.1740601


The factorial function is $f(x)=x!=\Gamma(x+1)$, monotonically increasing $\forall\space x\in\mathbb{N}$