2
$\begingroup$

This question already has an answer here:

I'm taking an intro to statistics and currently we are covering what a correlation coefficient is. Unfortunately the material does not explain why correlation coefficient is an indicator of linear relationship, it just says that it is. Can someone give an explanation to why correlation indicates linear relationship as opposed to quadratic or even cubic?

$\endgroup$

marked as duplicate by kjetil b halvorsen, Michael Chernick, Peter Flom Apr 30 '17 at 21:32

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 1
    $\begingroup$ Hint: Plug $a + bx$ rather than $y$ into the formula for Pearson correlation for $x, y$ to see what happens for such a case. $a$ will disappear and the magnitude of $b$ can be factored out of top and bottom. Then the correlation reduces to $+1$ for $ b > 0$, $0$ for $ b = 0$ and $-1$ for $b < 0$. $\endgroup$ – Nick Cox Feb 26 '16 at 18:59
  • $\begingroup$ Do your textbooks or notes actually define or explain what they mean by "linear relationship"? If so, what do they say it is? $\endgroup$ – whuber Feb 26 '16 at 19:18
  • $\begingroup$ @whuber No they don't, but as far as I understand linear relationship means, $y = ax+b$ $\endgroup$ – flashburn Feb 26 '16 at 20:22
  • $\begingroup$ Yes, that's correct. The problems begin when the data (or even a model) depart from that idealization. We have to decide what it means for them to be "approximately" linear and how to measure departures from linearity. There are many subtleties. For instance, if you think of the individual data $y_i$ as varying randomly from this ideal relationship or you think of $x_i$ as varying randomly, you get two different models and two different senses in which the data may appear to be nonlinear, even though both models are perfectly linear. $\endgroup$ – whuber Feb 26 '16 at 21:56
  • $\begingroup$ Also look at stats.stackexchange.com/questions/38856/… $\endgroup$ – kjetil b halvorsen Apr 30 '17 at 18:47
4
$\begingroup$

The correlation coefficient $\rho$ is intrinsically related to the linear regression coefficient, or the least squares estimated "slope", which is often called $\beta$. The formula relating these values is:

$$\beta = \rho \frac{\mbox{var} \left(Y \right)}{ \mbox{var} \left(X\right)} $$

So, if $X$ and $Y$ are centered and scaled, so that their variances are 1, (this means they are transformed into unitless quantities) then the regression slope is the correlation coefficient.

The least squares regression coefficient is often interpreted as a "first order trend" meaning that if the data take some nonlinear form, the slope gives you a "best fitting" line to that curve. This is intuitively useful, because only if a trend is highly curvilinear does the slope fails to show much strong relationship with the data. Any perfectly symmetrical trend about the mean of the $Y$ will have a slope equal to 0. Another interpretation of the least squares slope is an average derivative: that is, whatever the actual trend is between $X$ and $Y$ (imaging some meandering, wiggling trend), the first order trend or the least squares regression slope is what you get taking a weighted average of the instantaneous trend at each point.

I spoke a lot about $\beta$ when you asked about $\rho$. The relationship serves to show they are similar, and indeed inferring about one is equivalent to inferring about the other. I prefer the slope as a data summary measure. I look at the fact that it incorporates the units of $X$ and $Y$ as a very good thing, because it means the summary is contextually appropriate. For instance, I want to know how much I'd expect blood pressure to change with an extra 10mg / day dose of a hypertension medication, not simply the correlation between them.

$\endgroup$
  • $\begingroup$ I think a little more thought and discussion might be appropriate for this question. When the data are of the form $\{(x_i, y)\}$ for a constant $y$, then in the mathematical sense this is a perfectly linear relationship, but the correlation coefficient $\rho$ is zero. Consider, too, that a purely quadratic or cubic relationship can (and often does) produce a large value of $|\rho|$. Such examples leave us still wondering about the precise sense in which a correlation coefficient and "linear relationships" are related. $\endgroup$ – whuber Feb 26 '16 at 19:17
  • $\begingroup$ @whuber well some notes about first order trend, or average derivative might be worth adding. But it's generally known that a flat line has a slope of 0! $\endgroup$ – AdamO Feb 26 '16 at 19:31
  • $\begingroup$ And doesn't that show, very forcibly, that $\rho$ has some inadequacies as a measure of linearity, since a "flat line" is as linear as anything can ever get? $\endgroup$ – whuber Feb 26 '16 at 21:57

Not the answer you're looking for? Browse other questions tagged or ask your own question.