I am performing a regression on a large dataset that is fairly noisy. The line I am running in R is:

lmfit <- lm(predictVariable ~ dataSet[,1:10])

so I have 10 endogenous variables. The results are somewhat strange:

  1. significant t-statistics for all the coefficients

  2. $R^2 = 0.25$

But when I run a regression again on the fitted and actual variables, I get this:

run1 <- lm(predictVariable ~ lmfit$fitted.values)

The coefficient in front of fitted values is 1.0001 and the intercept is ~0. But now when i run it in the reverse way:

run1 <- lm(lmfit$fitted.values ~ predictVariable)

The coefficient is now 0.26 and the intercept is also ~0. Why are the coefficients in the last two regressions so different? Does this mean my endogenous variables are colinear, or that my fit is poor? When I plot the fitted.values versus the actual, I see that there are many actual values close to 0 but the fitted value is higher and lower. Can anyone help me understand these regression results?

  • 1
    $\begingroup$ One thing to realize about linear regression is that regressing $Y$ on $X$ is not the same as regressing $X$ on $Y$, see here. $\endgroup$ – gung Jun 8 '12 at 2:43
  • $\begingroup$ You shouldn't expect a simple relationship between $E(Y|X)$, which is estimated by the first model, and $E(X|Y)$, which is estimated by the second model, as @gung has pointed out. Also, can you please clarify what you mean by an "endogenous variable"? From the context it appears you mean it to refer to the predictor variables but I (and perhaps others) am not very familiar with that terminology. $\endgroup$ – Macro Jun 8 '12 at 2:46
  • $\begingroup$ I agree with gung and Macro. The two regressions are different problems. Also the slopes should of course be different. If Y=aX then X=Y/a. $\endgroup$ – Michael Chernick Jun 8 '12 at 9:59
  • $\begingroup$ Can you post the plots. $\endgroup$ – simmmons Jun 8 '12 at 13:15
  • $\begingroup$ Yes--gung your explanation makes a lot of sense. If I do a total least squares, then should the fitted coefficients be the the inverse of each other? $\endgroup$ – Sam Jun 10 '12 at 13:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.