Skip to main content
3 of 7
added 23 characters in body
dimitriy
  • 38.3k
  • 7
  • 84
  • 168

This is more of comment, but I wanted to include a graph and some code.

I think the statement "if two predictors are correlated and both are included in a model, one will be insignificant" is false if you mean "only one."

Here's my counterexample using a regression of body fat percentage on thigh circumference, skinfold thickness, and midarm circumference:

. webuse bodyfat, clear
(Body Fat)

. reg bodyfat thigh triceps midarm

      Source |       SS       df       MS              Number of obs =      20
-------------+------------------------------           F(  3,    16) =   21.52
       Model |  396.984607     3  132.328202           Prob > F      =  0.0000
    Residual |  98.4049068    16  6.15030667           R-squared     =  0.8014
-------------+------------------------------           Adj R-squared =  0.7641
       Total |  495.389513    19  26.0731323           Root MSE      =    2.48

------------------------------------------------------------------------------
     bodyfat |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       thigh |  -2.856842   2.582015    -1.11   0.285    -8.330468    2.616785
     triceps |   4.334085   3.015511     1.44   0.170    -2.058512    10.72668
      midarm |  -2.186056   1.595499    -1.37   0.190    -5.568362     1.19625
       _cons |   117.0844   99.78238     1.17   0.258    -94.44474    328.6136
------------------------------------------------------------------------------

. corr bodyfat thigh triceps midarm 
(obs=20)

             |  bodyfat    thigh  triceps   midarm
-------------+------------------------------------
     bodyfat |   1.0000
       thigh |   0.8781   1.0000
     triceps |   0.8433   0.9238   1.0000
      midarm |   0.1424   0.0847   0.4578   1.0000


. ellip thigh triceps, coefs plot(, yline(0, lcolor(gray)) xline(0, lcolor(gray)))

enter image description here

As you can see from the regression table, everything is insignificant.

The last Stata command graphs the confidence region for 2 of the regression coefficients (a 2 dimensional analog of the familiar confidence intervals). The confidence ellipse for the skinfold thickness and thigh circumference coefficients is long, narrow and tilted, reflecting the collinearity in the regressors. There's high negative covariance between the estimated coefficients. The ellipse covers parts of the vertical and the horizontal axes, which means that we cannot reject the individual hypotheses that the $\beta$s are zero, though we can reject the joint null that both are since the ellipse does not cover the origin. In other words, either thigh and triceps are relevant for body fat, but you can't determine which one is the culprit.

dimitriy
  • 38.3k
  • 7
  • 84
  • 168