Multicolinearity Test for Multiple Multivariate Regression

I have multiple independent variables and multiple dependent variables, some categorical and some quantitative. I have created a data sheet with dummy columns appropriate to each categorical variable.

My team and I have ran various tests with our data, including Multivariate Multiple Regression, and will need to re-do all of them in light of a multicolinearity test that will surely eliminate some variables. This is our last step before writing up a manuscript, so all and any help is deeply appreciated.

I have access to SPSS, SAS and R (though no experience with R). Multicolinearity tests are simple enough for multiple regression with SPSS, but I'm lost when it comes to multiple multivariate regression.

Any suggestions for how to test for multicolinearity for multivariate multiple regression?

Here is a link to some of our data:

Our dependent variables are the columns labeled with names of academic fields, all on the left-most side. Our independent variables are the columns to the right of those, from "Major Fields" to "Average ACT." To the right of that are the columns for the dummy variables. I've color coded the dummy columns to the variable to which they correspond.

Multicollinearity has nothing to do with the response variable!

(In fact, if I’m doing a linear regression, I hope for a strong linear relationship between the predictors and the response!)

Think about variance inflation factor. At no point do you consider the response variable.

If you know how to examine for multicollinearity in linear regression with one response variable, you know how to do it for multiple response variables. (Ditto for a binary response variable (e.g. logistic regression) or a count response variable (e.g. Poisson regression ), since only the predictors matter for this.)

You can perform a Principal Component Analysis (PCA) based on the correlation matrix of your continuous variables. PCA is a method of dimension reduction and hence it is usually applied to reduce the number of collinear variables, like in your case. Many softwares, such as PC-ORD, can perform it. But, I am not sure that you can do correlation analyses with your categorical or nominal variables at the same time. However, you can get some ideas on how to handle these nominal variables after you have reduced some of collinear continuous variables through PCA.

• Ah, I should of mentioned. We have done PCA analysis to two components. My statistician, however, thinks we still need to do some other multicolinearity test. Jun 6 '16 at 15:44

I have used the vif function from the car package in R to obtain the Variance Inflation Factor (VIF), which is a measure of multi-collinearity. You should be able to do the same with your multivariate analysis. Basically, once you obtain your multivariate model, you'll need to load the car package, simply write vif(name_of_your_model) in R and you'll obtain the VIF for each of your explanatory variable. Note that according to Zuur et al. (2009) Mixed Effects Models and Extensions in Ecology with R, if a predictor variable as a VIF ­> 5 it means that there's a multi-collinearity issue. You would have to exclude the predictor variable with the highest VIF, redo the vif test on the updated model and repeat these steps until all your predictor variables have a VIF < 5.