1
$\begingroup$

I was playing around with R built-in library mtcars and don't quite understand why I get following results.

attach(mtcars)
cor(mtcars[, 1:7])
            mpg        cyl       disp         hp        drat         wt        qsec
mpg   1.0000000 -0.8521620 -0.8475514 -0.7761684  0.68117191 -0.8676594  0.41868403
cyl  -0.8521620  1.0000000  0.9020329  0.8324475 -0.69993811  0.7824958 -0.59124207
disp -0.8475514  0.9020329  1.0000000  0.7909486 -0.71021393  0.8879799 -0.43369788
hp   -0.7761684  0.8324475  0.7909486  1.0000000 -0.44875912  0.6587479 -0.70822339
drat  0.6811719 -0.6999381 -0.7102139 -0.4487591  1.00000000 -0.7124406  0.09120476
wt   -0.8676594  0.7824958  0.8879799  0.6587479 -0.71244065  1.0000000 -0.17471588
qsec  0.4186840 -0.5912421 -0.4336979 -0.7082234  0.09120476 -0.1747159  1.00000000

So I see that mpg (dependent variable) is correlated with six variables that I choose. Plot looks like this enter image description here

If I regress mpg on, for example, hp then I have signigificant coefficient. However, if I include all variables in equation than I end up with insignificant coefficients for almost all betas:

> summary(lm(mpg ~ hp))

Call:
lm(formula = mpg ~ hp)

Residuals:
    Min      1Q  Median      3Q     Max 
-5.7121 -2.1122 -0.8854  1.5819  8.2360 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 30.09886    1.63392  18.421  < 2e-16 ***
hp          -0.06823    0.01012  -6.742 1.79e-07 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 3.863 on 30 degrees of freedom
Multiple R-squared:  0.6024,    Adjusted R-squared:  0.5892 
F-statistic: 45.46 on 1 and 30 DF,  p-value: 1.788e-07

> summary(lm(mpg ~ cyl + disp+ hp + drat + wt + qsec))

Call:
lm(formula = mpg ~ cyl + disp + hp + drat + wt + qsec)

Residuals:
    Min      1Q  Median      3Q     Max 
-3.9682 -1.5795 -0.4353  1.1662  5.5272 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)   
(Intercept) 26.30736   14.62994   1.798  0.08424 . 
cyl         -0.81856    0.81156  -1.009  0.32282   
disp         0.01320    0.01204   1.097  0.28307   
hp          -0.01793    0.01551  -1.156  0.25846   
drat         1.32041    1.47948   0.892  0.38065   
wt          -4.19083    1.25791  -3.332  0.00269 **
qsec         0.40146    0.51658   0.777  0.44436   
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 2.557 on 25 degrees of freedom
Multiple R-squared:  0.8548,    Adjusted R-squared:   0.82 
F-statistic: 24.53 on 6 and 25 DF,  p-value: 2.45e-09

Why this is the case? Intuitively, mpg can be described by chosen variables. Also knowing about omitted variable bias, it is better to include all potential variables into equation. What conclusion should I make from this results?

$\endgroup$
  • $\begingroup$ "it is better to include all potential variables into equation." No. Consider collinearity. Calculate the variance inflation factors. $\endgroup$ – Roland Oct 27 '16 at 11:14
1
$\begingroup$

When you fit a model with several predictors the estimates and their standard errors are for the effect of that variables over and above the effect of the others. If the predictors are related then although together they do a good job each in turn may not add much to the others. You have, wisely, looked at the correlations between them and this gives you a clue to the answer to your question as most of the coefficients are fairly large in magnitude.

$\endgroup$
  • $\begingroup$ Thanks for your answer. Could you explain in mathematical terms why this happens? $\endgroup$ – tosik Oct 27 '16 at 13:59
0
$\begingroup$

Another important thing is that mpg is a discrete variable; so think if does it have sense to analyze the correlation between a continuous variable and a discrete one.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.