2
$\begingroup$

Lets consider normalized variables X and Y. Slope of a lm(Y~X) is Cor(Y,X)*sd(Y)/sd(X) and for lm(X~Y) its Cor(X,Y)*sd(X)/sd(Y). Since sd(X) = sd(Y) = 1. The slopes of lm(Y~X) and lm(X~Y) are always bound to be equal.

That would mean the regression lines for both of the models will be, $Y = mX$ and $X = mY$.

Lets take an example. 

x = rnorm(20)
y = rnorm(20)
x = (x-mean(x))/sd(x)
y = (y-mean(y))/sd(y)
lm(x~y)

Call:
lm(formula = x ~ y)

Coefficients:
(Intercept)            y  
  4.333e-17   -1.272e-01  

lm(y~x)

Call:
lm(formula = y ~ x)

Coefficients:
(Intercept)            x  
 -4.333e-17     -1.272e-01

Suppose we have two normalized variables X and Y as shown above. They have Cor(X,Y) = Cor(Y,X) = -0.127.

If I use X as a predictor variable of Y, I get Y = -0.127X; then, I expect X = -1/0.127 Y.

If I use X as a predictor variable of Y, I get Y = -0.127X;

It looks like both statements are inconsistent. How to interpret this result?

Improve this question
$\endgroup$
4
  • 4
    $\begingroup$ That poor, forgotten intercept... $\endgroup$
    – shadowtalker
    Commented Jun 14, 2015 at 19:18
  • 2
    $\begingroup$ Apologies for poor editing and typos.. For normalized variables the intercept will be zero. $\endgroup$
    – emperorspride188
    Commented Jun 14, 2015 at 20:07
  • 3
    $\begingroup$ Search your references for "regression to the mean." An extreme example is when $X$ and $Y$ are uncorrelated, for then the standardized regressions are $X = 0$ and $Y = 0$. That's perfectly consistent, even though those two lines are as different as possible! $\endgroup$
    – whuber ♦
    Commented Jun 14, 2015 at 20:32
  • $\begingroup$ Another useful exercise is to consider what the standardisation does, e.g. with 2 points. (I guess it is tempting to think that standardisation is a "neutral" transformation for linear regression) $\endgroup$
    – P.Windridge
    Commented Jun 14, 2015 at 20:49

0

Reset to default

Browse other questions tagged or ask your own question.