Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I am performing a regression on a large dataset that is fairly noisy. The line I am running in R is:

lmfit <- lm(predictVariable ~ dataSet[,1:10])

so I have 10 endogenous variables. The results are somewhat strange:

  1. significant t-statistics for all the coefficients

  2. $R^2 = 0.25$

But when I run a regression again on the fitted and actual variables, I get this:

run1 <- lm(predictVariable ~ lmfit$fitted.values)

The coefficient in front of fitted values is 1.0001 and the intercept is ~0. But now when i run it in the reverse way:

run1 <- lm(lmfit$fitted.values ~ predictVariable)

The coefficient is now 0.26 and the intercept is also ~0. Why are the coefficients in the last two regressions so different? Does this mean my endogenous variables are colinear, or that my fit is poor? When I plot the fitted.values versus the actual, I see that there are many actual values close to 0 but the fitted value is higher and lower. Can anyone help me understand these regression results?

share|improve this question
1  
One thing to realize about linear regression is that regressing $Y$ on $X$ is not the same as regressing $X$ on $Y$, see here. –  gung Jun 8 '12 at 2:43
    
You shouldn't expect a simple relationship between $E(Y|X)$, which is estimated by the first model, and $E(X|Y)$, which is estimated by the second model, as @gung has pointed out. Also, can you please clarify what you mean by an "endogenous variable"? From the context it appears you mean it to refer to the predictor variables but I (and perhaps others) am not very familiar with that terminology. –  Macro Jun 8 '12 at 2:46
    
I agree with gung and Macro. The two regressions are different problems. Also the slopes should of course be different. If Y=aX then X=Y/a. –  Michael Chernick Jun 8 '12 at 9:59
    
Can you post the plots. –  simmmons Jun 8 '12 at 13:15
    
Yes--gung your explanation makes a lot of sense. If I do a total least squares, then should the fitted coefficients be the the inverse of each other? –  Sam Jun 10 '12 at 13:37
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.