How does the correlation coefficient differ from regression slope?

Question

I would have expected the correlation coefficient to be the same as a regression slope (beta), however having just compared the two, they are different. How do they differ - what different information do they give?

Answer 1

Assuming you're talking about a simple regression model $$Y_i = \alpha + \beta X_i + \varepsilon_i$$ estimated by least squares, we know from wikipedia that $$ \hat {\beta} = {\rm cor}(Y_i, X_i) \cdot \frac{ {\rm SD}(Y_i) }{ {\rm SD}(X_i) } $$ Therefore the two only coincide when ${\rm SD}(Y_i) = {\rm SD}(X_i)$. That is, they only coincide when the two variables are on the same scale, in some sense. The most common way of achieving this is through standardization, as indicated by @gung.

The two, in some sense give you the same information - they each tell you the strength of the linear relationship between $X_i$ and $Y_i$. But, they do each give you distinct information (except, of course, when they are exactly the same):

• The correlation gives you a bounded measurement that can be interpreted independently of the scale of the two variables. The closer the estimated correlation is to $\pm 1$, the closer the two are to a perfect linear relationship. The regression slope, in isolation, does not tell you that piece of information.

• The regression slope gives a useful quantity interpreted as the estimated change in the expected value of $Y_i$ for a given value of $X_i$. Specifically, $\hat \beta$ tells you the change in the expected value of $Y_i$ corresponding to a 1-unit increase in $X_i$. This information can not be deduced from the correlation coefficient alone.

Answer 2

With simple linear regression (i.e., only 1 covariate), the slope $\beta_1$ is the same as Pearson's $r$ if both variables were standardized first. (For more information, you might find my answer here helpful.) When you are doing multiple regression, this can be more complicated due to multicollinearity, etc.

Answer 3

The correlation coefficient measures the "tightness" of linear relationship between two variables and is bounded between -1 and 1, inclusive. Correlations close to zero represent no linear association between the variables, whereas correlations close to -1 or +1 indicate strong linear relationship. Intuitively, the easier it is for you to draw a line of best fit through a scatterplot, the more correlated they are.

The regression slope measures the "steepness" of the linear relationship between two variables and can take any value from $-\infty$ to $+\infty$. Slopes near zero mean that the response (Y) variable changes slowly as the predictor (X) variable changes. Slopes that are further from zero (either in the negative or positive direction) mean the response changes more rapidly as the predictor changes. Intuitively, if you were to draw a line of best fit through a scatterplot, the steeper it is, the further your slope is from zero.

So the correlation coefficient and regression slope MUST have the same sign (+ or -), but will not have the same value.

For simplicity, this answer assumes simple linear regression.

Answer 4

Pearson's correlation coefficient is dimensionless and scaled between -1 and 1 regardless of the dimension and scale of the input variables.

If (for example) you input a mass in grams or kilograms, it makes no difference to the value of $r$, whereas this will make a tremendous difference to the gradient/slope (which has dimension and is scaled accordingly ... likewise, it would make no difference to $r$ if the scale is adjusted in any way, including using pounds or tons instead).

A simple demonstration (apologies for using Python!):

import numpy as np
x = [10, 20, 30, 40]
y = [3, 5, 10, 11]
np.corrcoef(x,y)[0][1]
x = [1, 2, 3, 4]
np.corrcoef(x,y)[0][1]

shows that $r = 0.969363$ even though the slope has been increased by a factor of 10.

I must confess it's a neat trick that $r$ comes to be scaled between -1 and 1 (one of those cases where the numerator can never have absolute value greater than the denominator).

As @Macro has detailed above, slope $b = r(\frac{\sigma_{y}}{\sigma_{x}})$ , so you are correct in intuiting that Pearson's $r$ is related to the slope, but only when adjusted according to the standard deviations (which effectively restores the dimensions and scales!).

At first I thought it odd that the formula seems to suggest a loosely fitted line (low $r$) results in a lower gradient; then I plotted an example and realised that given a gradient, varying the "looseness" results in $r$ decreasing but this is offset by a proportional increase in $\sigma_{y}$.

In the chart below, four $x,y$ datasets are plotted:

1. the results of $y=3x$ (so gradient $b=3$, $r=1$, $\sigma_{x}=2.89$, $\sigma_{y}=8.66$) ... note that $\frac{\sigma_{y}}{\sigma_{x}}=3 $
2. the same but varied by a random number, with $r = 0.2447$, $\sigma_{x}=2.89$, $\sigma_{y}=34.69$, from which we can compute $b= 2.94 $
3. $y=15x$ (so $b=15$ and $r=1$, $\sigma_{x}=0.58$, $\sigma_{y}=8.66$)
4. the same as (2) but with reduced range $x$ so $ b= 14.70$ (and still $r = 0.2447$, $\sigma_{x}=0.58$, $\sigma_{y}=34.69$)

It can be seen that variance affects $r$ without necessarily affecting $b$, and units of measure can affect scale and thus $b$ without affecting $r$