# Choosing Predictors in Multiple Regression

I am planning regression analyses and present this (hypothetical) scenario to communicate my query.

I am interested in the effect of 2 different measures ('IQ' and 'SPQ') on dependent variable 'performance'. Variance inflation factor shows that there is no problematic collinearity, but the two variables are correlated (r = ~0.6).

I run separate linear regressions predicting performance:

1. performance ~ IQ
2. performance ~ SPQ

SCENARIO 1: Analyses 1) and 2) show that IQ and SPQ significantly predict performance. I therefore decide to run a multiple regression to isolate the effects of IQ when controlling for SPQ (or vice versa):

1. performance ~ IQ + SPQ

QUESTION 1- I believe that I need to run this multiple regression even if the predictors are only weakly correlated?

SCENARIO 2: Analyses 1) and 2) show that only IQ significantly predicts performance.

QUESTION 2- Am I correct that I therefore do not need to run analysis 3) as SPQ does not significantly predict performance? Or perhaps this analysis would still be useful in showing that IQ has an effect independent of SPQ?

You write

SCENARIO 1: Analyses 1) and 2) show that IQ and SPQ significantly predict performance. I therefore decide to run a multiple regression to isolate the effects of IQ when controlling for SPQ (or vice versa):

This is not a good reason to decide to run the multiple regression. It is known as bivariate screening and the results are problematic.

QUESTION 1- I believe that I need to run this multiple regression even if the predictors are only weakly correlated?

No. You need to run this regression if you are interested in the effects of the IQ and SPQ on performance while controlling for the other variables. If IQ and SPQ are completely orthogonal, then putting them in the same equation won't affect the parameter estimate of the other one (e.g. adding SPQ won't affect the parameter estimate for IQ. But that's rarely the case, and not in your case. Putting both in one regression also means you have one regression to interpret rather than two.

Adding another variable can also reduce noise and, therefore, the standard error of the parameter estimate of the other variable.

SCENARIO 2: Analyses 1) and 2) show that only IQ significantly predicts performance.

QUESTION 2- Am I correct that I therefore do not need to run analysis 3) as SPQ does not significantly predict performance? Or perhaps this analysis would still be useful in showing that IQ has an effect independent of SPQ?

No. Significance is not the issue here. Indeed, adding SPQ could affect the parameter estimate for IQ, and its standard error.

The best thing to do is to decide, before doing any analysis, what analyses you will do. Frank Harrell has written a lot about this, both on here and in his book Regression Modeling Strategies. Personally, I am not absolutely rigid about this, but it's a goal. And, if you adjust your analysis based on the results of earlier analysis, you have to be very careful. One good method is to divide the data into train and test sets (and possibly also a validate set).

• 'You need to run this regression if you are interested in the effects of the IQ and SPQ on performance while controlling for the other variables.' Do you mean in the case of 3 variables? Commented Jan 9 at 15:22
• It could be any number of variables. Your model (at least, from what you wrote) has only two. Commented Jan 9 at 15:26
• I'm aiming to identify the effect of SPQ independent of other variables. However, some of my variables are correlated, so I felt that placing them all in a multiple regression would not be the best idea? Commented Jan 9 at 15:26
• You said that you looked at VIFs and that they indicated no problem. Correlated independent variables are only problematic some of the time. VIF (and other collinearity diagnostics) are ways of identifying those times. Commented Jan 9 at 15:29
• The vifs are not problematic (< 3) but some of the variables are quite highly correlated (r=0.67). Would this not still mean that the variance explained by one variable could be attributed to the other variable? Commented Jan 9 at 15:35