How can I test whether 2 predictors/covariates in MLR are really useful in prediction using R? I reckon this is something to do with multicollinearity. So would the CAR library function , VIF() be appropriate? or can I just do it using simple lm() + summary()? Thanks.

Update: Is this correct?

> model1 = lm( Y ~ X1)
> model2 = lm( Y ~ X1 + X2)
> anova(model1, model2)
Model 1: Y ~ X1
Model 2: Y ~ X1 + X2
  Res.Df     RSS Df Sum of Sq      F  Pr(>F)  
1     36 0.31814                              
2     35 0.27624  1  0.041904 5.3093 0.02727 *

ok so how to interpret the result. Pr=0.02 < 0.05 so can i say X2 has an effect and is useful as a predictor ? how about the F value? If i look up F(0.95,1,35) in F table then, F=5.31 > 4.17 = should include X2 ? is this correct?


  • $\begingroup$ Without some indication of quite what you intend by 'really useful' this is going to be tricky to answer. $\endgroup$ – Glen_b Apr 5 '13 at 8:34

If the two predictors are highly collinear you'll find that a model with either of them has about the same coefficient of determination ($R^2$) & that including both only marginally improves it. Moreover the standard errors for the regression coefficients become very large - this is what variance inflation factors measure. So you don't need VIFs to detect multicollinearity, but they quantify it & come in handy when there's a lot of predictors to consider.

The test you describe justifies including $X_2$ in your model, given that you've already included $X_1$, but doesn't address the question of whether $X_2$ alone would be a sufficiently good predictor - it perhaps needn't, as you know what $X_1$ & $X_2$ really refer to.

Model selection is quite a big topic (with a tag on this site) &, as Glen was getting at, it's important to think carefully about your aims, & what you already know about what it is you're studying. In response to the question in your comment, that's the right idea, but (1) the confidence interval around the coefficient is more informative than the p-value alone (especially important is to see if it's too broad to allow you to distinguish substantively different coefficient values); & (2) if the predictors are highly collinear you can't be sure whether $X_1$ is only any good because it correlates strongly with $X_2$ or vice versa, which becomes quite relevant if you have to make predictions for a population in which $X_1$ & $X_2$ aren't correlated.

| cite | improve this answer | |
  • $\begingroup$ thanks. If i want to check whether X1 alone or X2 alone can be a good predictor, i just need to do anova(modelwithX1), anova(modelwithX2) and then read their P-value right? $\endgroup$ – dorothy Apr 5 '13 at 9:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.