I did factor analysis and found three factors. To examine if which factors significantly affected a certain dependent variable, I added all three factors to a regression model. The correlation coefficients among the three factors are 0.7 to 0.8. The correlation between the factors and the dependent variable was about 0.30 at a 0.1 significant level. The result was that R-squared as the model fit was significant, but each standardization of the coefficients was not. However, using simple regression, all were significant. VIF was below 5 so there is no multicollinearity, I guess. The significant level was the same even after removing one of the factors. How should I analyze this? A stepwise method should be better?
1
-
$\begingroup$ This may be of help. $\endgroup$– DayneOct 21, 2020 at 5:35
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
A stepwise method should not be better. To a first approximation, stepwise regression should never be used (cf., Algorithms for automatic model selection).
You have multicollinearity. The rule of thumb that you don't have problematic multicollinearity until the VIFs are >10 is just a rule of thumb. Your variables are still strongly correlated with each other. My guess is that you have relatively few data, such that the variables are not significant even though the VIFs are <5. Let's say the VIFs = 4 for all variables. That means the variances of the sampling distributions are 4x larger than they would be with uncorrelated variables. That means that the standard deviations of the sampling distributions (i.e., your standard errors) are 2x larger than they would be. If your N is low enough, a significant variable could become non-significant by doubling its SE.
This is not an uncommon situation. It is harder to tell which variable is doing what when they are all very similar. In your case, you can conclude that some combination of your variables (maybe just one, or some two of them, or all of them) is related to the dependent variable, but you don't know which.
answered Oct 21, 2020 at 18:08
-
$\begingroup$ Thank you. I guess now that rather than multicollinearity, I doubt that there is small correlation between independent and dependent variables. It is 0.20-25. Thats why I guess the coefficients are not significant.. For the advice that independent variables should be more, I think there would be cases where only two variables were added to check a dependent variable so depending on research it should be ok. No? $\endgroup$– TotoOct 22, 2020 at 11:06
-
$\begingroup$ There's not much to be done at this point, @Toto. To differentiate between highly correlated variables, you'll need more data. $\endgroup$ Oct 22, 2020 at 11:37
-
-
$\begingroup$ @Toto, no I mean a larger N. For example, if N people fill out a questionnaire, & the questions are factor analyzed into 3 factors that are used in the multiple regression model, you need to replicate the study with a larger number of people answering the questions. $\endgroup$ Oct 22, 2020 at 14:04