1
$\begingroup$

I really hope you can help! I'm in the last stages of my PhD. My supervisor is keen on including all variables in the multiple regressions I am running. Some of the scales are intercorrelated (some with as high as r=.80). Is there a reason to still include them all?

I've seen some other posts and they mention multicollinearity, but if I'm looking at finding the best predictors out of a related list of possible ones (that are correlated) can I still include them all in the regression to do that? I have mainly non-significant regressions, and some significant beta values. How am I supposed to interpret them meaningfully?

From what I can interpret from other posts, it looks like I should still include them regardless and maybe discuss it after? However, if the regression isn't significant then does it just mean I can't read too much into the significant predictors? Would they still be worth looking at in the future in a follow up? I seem to have low power (n=50).

My standard errors are a maximum of .5 in some of the regression models, but usually around .2 or .3 in others. Any advice would be greatly appreciated.

Improve this question
$\endgroup$
4
  • $\begingroup$ So you're saying that the F-statistic for your model is insignificant?? Because you often see the exact opposite behavior when multicollinearity is an issue--your F-statistic will be really high while none of your estimated coefficients will have significant p-values. $\endgroup$
    – Steve S
    Nov 16, 2014 at 13:03
  • $\begingroup$ Hi thanks you for the answers! Yes the F is not significant and rather low, but I'm still getting significant predictors. Should I not be getting no significant predictors in a model that is not significant or does that mean something? $\endgroup$
    – user59988
    Nov 16, 2014 at 13:31
  • $\begingroup$ A low F statistic suggests there may be little evidence of any linear relationship. It is instructive to generate random data and study the betas and p-values that result. For instance randomly permute just the response variables and rerun the analysis. If the results are no worse, then you had nothing to begin with. $\endgroup$
    – whuber ♦
    Nov 16, 2014 at 14:48
  • $\begingroup$ Ok thank you, I'm just wondering then - how could I interpret the significant Betas with no significant F stat? For example if I re-run the regressions with just the significant ones, on some dvs the regression becomes significant, on other dvs it doesn't. However, my supervisor doesn't want me to do that. Is there any way I can explain them in more detail? She said that there might be a problem in the data if the F is not significant but some Betas are. I'm just not sure how to write it up meaningfully. Any help would be really appreciated $\endgroup$
    – user59988
    Nov 16, 2014 at 15:53

2 Answers 2

Reset to default
1
$\begingroup$

Including variables in your multiple regressions is something that depends on your hypothesis and what you are testing. But you can check the variance inflation factor (VIF) that is used as an indicator of multicollinearity. If VIFs are less that 10, means multicollinearity is not a problem. If VIFs for two variables is 10 or higher then you have to keep just one of those variables and eliminate the other one. Hope this help.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Hi, thank you! Yes maybe that's why he'd like me to keep them in I guess. Do you know how I can calculate or get SPSS to calculate this VIF value? $\endgroup$
    – user59988
    Nov 16, 2014 at 13:32
  • 1
    $\begingroup$ VIF is a diagnostic tool, not a rule or law. You don't have to do anything at all when it exceeds 10. $\endgroup$
    – whuber ♦
    Nov 16, 2014 at 14:42
  • $\begingroup$ Ok thank you! I will try to find it on the options now : ) It would be good to know if multicollinearity may be a problem even if I don't have to change anything, thanks for the tip $\endgroup$
    – user59988
    Nov 16, 2014 at 15:54
  • $\begingroup$ As mentioned by @whuber (+1) the VIF cannot be taken as a rule or law, but as a rule of thumb, we can say that it is recommended to keep just one of those variables if the VIF turns out to be 10 or higher. $\endgroup$
    – javlacalle
    Nov 16, 2014 at 20:34
1
$\begingroup$

There are several options in dealing with multicollinearity. Excluding highly correlated variables, using Principal Components Analysis to create uncorrelated predictors, using shrinkage methods (i.e. ridge regression), excluding common variance from one of the offending predictors but keeping it in another (by regressing one on another and keeping the residual as a predictor in the main model), and so forth. Using Bayesian methods is yet another approach.

An excellent paper in this regard can be found here: http://srmo.sagepub.com/view/the-sage-encyclopedia-of-social-science-research-methods/n586.xml

The exact choice depends on the specifics of your research. The paper should be a good starting point.

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.