4
$\begingroup$

In order to measure precisely the influence of one variable $Y$ over $X$ I wanted to use the mutual information because so far I believed that correlation coefficient (Pearson) was only limited to linear relation.

But I realized that even in non-linear cases it gives good results.

Example in Matlab:

$X$ is a uniform random variable and $I$ is the range of $X$

$Y=X^{2},I=[0,100]: \rho=0.96$

$Y=\sqrt{X} ,I=[0,100]: \rho=0.94$

$Y=exp(X),I=[0,10]: \rho=0.70$

$Y=log(X),I=[0,100]: \rho=0.84$

Why the correlation is quite high in all those non-linear cases? I am beginning to think that correlation is only weak when the monotony of $X$ and $Y$ is not the same. Otherwise it is not so bad.

Do some people have a point of view on this?

Improve this question
$\endgroup$
7
  • $\begingroup$ What is the meaning of $I=[0,100]$? For which values of $X$ did you calculate the correlation coefficient? $\endgroup$
    – Alecos Papadopoulos
    Commented May 22, 2014 at 10:04
  • $\begingroup$ "when the mononty is changing": I imagine that you want to make some point about monotonicity, or the lack of it, but I didn't feel confident at identifying exactly what you mean, so I left that unedited. Regardless of what you meant. it is easy to manufacture, or indeed find, cases with nonlinear and non-monotonic relationships where correlation is exactly or practically zero. $\endgroup$
    – Nick Cox
    Commented May 22, 2014 at 11:04
  • $\begingroup$ I is the range where $X$ takes its values. @NickCox I mean I feel like when the monotony of $X$ is following the monotony of $Y$ we often find a high correlation coefficient regardless the type of relation (linear or non linear) $\endgroup$
    – Gandhi91
    Commented May 22, 2014 at 11:32
  • $\begingroup$ Sorry, but what you mean is completely unclear to me. It's the relation between the variables we are looking at. See e.g. en.wikipedia.org/wiki/Monotonic_function What precisely do you mean by the "monotony" of an individual variable? $\endgroup$
    – Nick Cox
    Commented May 22, 2014 at 12:04
  • 1
    $\begingroup$ "monotony" is a standard English word but it does not, so far as I know, have the mathematical meaning you seem to think it has. Monotony changing is just a matter of not being monotonic, or of being non-monotonic. I don't think this question is getting clearer or more helpful to anyone else. If the point is that monotonic nonlinear relationships can be associated with high (Pearson) correlation, a short answer is Indeed; and a slightly longer answer is that your teachers or textbooks were at fault if they didn't emphasise that. $\endgroup$
    – Nick Cox
    Commented May 22, 2014 at 12:15

2 Answers 2

Reset to default
6
$\begingroup$

A couple of monotonic examples in R (but very similar code works in Matlab):

> x=(1:100)
> y=exp(x)
> z=log(x)
> cor(z,y,method="kendall")
[1] 1
> cor(z,y,method="spearman")
[1] 1
> cor(z,y)
[1] 0.1549211

That's pretty low!

It's quite possible to make it much lower:

> cor(xxx,yyy,method="spearman")
[1] 1
> cor(xxx,yyy,method="kendall")
[1] 1
> cor(xxx,yyy)
[1] 0.010647

and in fact that very low Pearson correlation is caused by only a single outlier in x and a single outlier in y:

> cor(xxx[2:99],yyy[2:99])
[1] 1

So even when 98% of the points lie on a perfectly straight line, and the relationship is perfectly monotonic, the Pearson correlation can still be really close to zero.

In the case where $x$ is restricted to take the values 1, 2, ..., 100, the correlation can still be very low. Here's an example:

 cor(1:100,exp(1:100))
[1] 0.252032

I'm not sure how you got 0.7 - perhaps I misunderstood. I have another example with equispaced $x$ that is about 0.175.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I'm not sure to understand. In your first code you compute $corr(exp(X),log(X))$ I agree with the low coefficient. But it's not the example I was taking. In your last examples, I don't know what is 'xxx' and correlation is sensible to outliers, I agree. But my question was: why in the example of non linear cases I gave (without outliers) Pearson correlation is high? $\endgroup$
    – Gandhi91
    Commented May 22, 2014 at 11:47
  • $\begingroup$ If you limit $x$ to be equispaced (was that the intent?), then the minimum correlation is higher (but still a good deal lower than in your examples; I've included one at about 0.25. but it's possible to get quite a bit smaller than that). I didn't tell you the content of the second set of variables deliberately but only a general sense of how to obtain them - sometimes it's fun to discover on ones own. $\endgroup$
    – Glen_b
    Commented May 22, 2014 at 11:52
1
$\begingroup$

The more a function deviates from linearity, the smaller the correlation coefficient is.

Try this one:

> cor(1:100,factorial(1:100))
[1] 0.1740601

The factorial function is $f(x)=x!=\Gamma(x+1)$, monotonically increasing $\forall\space x\in\mathbb{N}$

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.