2
$\begingroup$

I am planning regression analyses and present this (hypothetical) scenario to communicate my query.

I am interested in the effect of 2 different measures ('IQ' and 'SPQ') on dependent variable 'performance'. Variance inflation factor shows that there is no problematic collinearity, but the two variables are correlated (r = ~0.6).

I run separate linear regressions predicting performance:

  1. performance ~ IQ
  2. performance ~ SPQ

SCENARIO 1: Analyses 1) and 2) show that IQ and SPQ significantly predict performance. I therefore decide to run a multiple regression to isolate the effects of IQ when controlling for SPQ (or vice versa):

  1. performance ~ IQ + SPQ

QUESTION 1- I believe that I need to run this multiple regression even if the predictors are only weakly correlated?

SCENARIO 2: Analyses 1) and 2) show that only IQ significantly predicts performance.

QUESTION 2- Am I correct that I therefore do not need to run analysis 3) as SPQ does not significantly predict performance? Or perhaps this analysis would still be useful in showing that IQ has an effect independent of SPQ?

$\endgroup$

1 Answer 1

3
$\begingroup$

You write

SCENARIO 1: Analyses 1) and 2) show that IQ and SPQ significantly predict performance. I therefore decide to run a multiple regression to isolate the effects of IQ when controlling for SPQ (or vice versa):

This is not a good reason to decide to run the multiple regression. It is known as bivariate screening and the results are problematic.

QUESTION 1- I believe that I need to run this multiple regression even if the predictors are only weakly correlated?

No. You need to run this regression if you are interested in the effects of the IQ and SPQ on performance while controlling for the other variables. If IQ and SPQ are completely orthogonal, then putting them in the same equation won't affect the parameter estimate of the other one (e.g. adding SPQ won't affect the parameter estimate for IQ. But that's rarely the case, and not in your case. Putting both in one regression also means you have one regression to interpret rather than two.

Adding another variable can also reduce noise and, therefore, the standard error of the parameter estimate of the other variable.

SCENARIO 2: Analyses 1) and 2) show that only IQ significantly predicts performance.

QUESTION 2- Am I correct that I therefore do not need to run analysis 3) as SPQ does not significantly predict performance? Or perhaps this analysis would still be useful in showing that IQ has an effect independent of SPQ?

No. Significance is not the issue here. Indeed, adding SPQ could affect the parameter estimate for IQ, and its standard error.

The best thing to do is to decide, before doing any analysis, what analyses you will do. Frank Harrell has written a lot about this, both on here and in his book Regression Modeling Strategies. Personally, I am not absolutely rigid about this, but it's a goal. And, if you adjust your analysis based on the results of earlier analysis, you have to be very careful. One good method is to divide the data into train and test sets (and possibly also a validate set).

$\endgroup$
7
  • $\begingroup$ 'You need to run this regression if you are interested in the effects of the IQ and SPQ on performance while controlling for the other variables.' Do you mean in the case of 3 variables? $\endgroup$
    – SilvaC
    Commented Jan 9 at 15:22
  • $\begingroup$ It could be any number of variables. Your model (at least, from what you wrote) has only two. $\endgroup$
    – Peter Flom
    Commented Jan 9 at 15:26
  • $\begingroup$ I'm aiming to identify the effect of SPQ independent of other variables. However, some of my variables are correlated, so I felt that placing them all in a multiple regression would not be the best idea? $\endgroup$
    – SilvaC
    Commented Jan 9 at 15:26
  • 1
    $\begingroup$ You said that you looked at VIFs and that they indicated no problem. Correlated independent variables are only problematic some of the time. VIF (and other collinearity diagnostics) are ways of identifying those times. $\endgroup$
    – Peter Flom
    Commented Jan 9 at 15:29
  • $\begingroup$ The vifs are not problematic (< 3) but some of the variables are quite highly correlated (r=0.67). Would this not still mean that the variance explained by one variable could be attributed to the other variable? $\endgroup$
    – SilvaC
    Commented Jan 9 at 15:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.