VIF understanding - does only >4 variables are multi-collinear and others are not?

Question

I am trying to understand if there will be multicollinearity between few variables or not. I took a sample data and tried to see the Variance Influencing Factor results - in general vif > 4 indicates multicollinearity.

From the below can I say that only Q6,Q5,Q7 are multicollinear and the rest are not?

Variables      VIF

1         Q1 3.294284

2         Q2 2.500329

3         Q3 3.229811

4         Q4 1.498705

5         Q5 4.833235

6         Q6 5.798955

7         Q7 4.183958

8         Q8 3.201985

9         Q9 3.159585

10        Q10 2.824077

Can I pass Q5, Q6, Q7 to PCA and take that component and raw Q1,Q2,Q3,Q4,Q8,Q9,Q10 and run my regression? Does this makes sense ?

Answer 1

The VIF for a given predictor variable tells you to what degree that variable is correlated with a linear combination of all the other predictors. This explains VIF pretty well.

So, you don't know for sure that Q5, Q6, and Q7 are the only predictors causing multicollinearity in your model, but removing the predictors with a high VIF one at a time and re-running the model can help you figure out which predictors would be most beneficial to remove.

If you have some understanding of what these variables represent that can help you decide which ones to keep in your model.

Answer 2

If you use a >4 threshold you could indeed say that Q5, Q6, Q7 are multicollinear and the other ones are ok. >4 is the most conservative threshold out there. As indicated many use >5 or even >10.

I think your solution is awkward. Running a PCA model on just those three variables will not work well. PCA is better catered when you have a bunch of variables that are multicollinear and you want to reduce them to just three indexes-like variables (3 principal components).

If you cared to, you could run a PCA with all 10 variables and derive the three principal component-variables that would pretty much retain the information in your model. However, PCA models get very opaque as the principal components often are difficult to interpret. So, I do not necessarily recommend going the PCA route.

I think your solution is rather qualitative in nature. First, look at your sample. Unless your sample is reasonably large (let's say over several hundred data points), I suspect you have way too many variables in this model. So, focus on the 5 variables that have the most explanatory meaning for what you are trying to estimate and go with that. Obviously, the Q5/Q6/Q7 variables appear somewhat redundant. You probably need to keep only one of those or maybe none at all.

So, the key is really to focus on what your variables mean and which one makes the most sense to keep in the model in order to explain your dependent variable.

Answer 3

In my point of view, multicollinearity is a problem if a vif value is bigger 10. But, if you think there is a problem, you should delete one of the variables with the high vif values and calculate the regression again. Perhaps, you find just one variable that is responsible for the high vif values. Then you should delete this variable. Otherwise, with pca you cannot interpret the parameters and you will lose information as well.