# Multicollinearity in an ordinal regression model?

I have a model with an ordinal DV and a few IVs that are categorical (nominal and ordinal) as well as one continuous variable. I recoded all the categorical variables with 3 or more categories into dummies to run the colinearity test. I have one variable (5-point likert scale, ordinal) that showed 2 of the 4 categories with VF>10. DO you know which is the right way to proceed? Should I erase the whole variable, just one category (randomly...)? I am using SPSS.

• Does your model make sense with the variables as they are? If so, why worry? Aug 29 '16 at 13:13

Further information given in comments by the OP suggest that the problem here is separation or quasi-separation since 85+% of the cells formed by a complete cross-classification are zeroes.

To answer the original question posed first: the finding of collinearity in the model is not necessarily a red flag as it is sometimes treated. It may not even be an orange alert either. It is conveying important information about the data-set which needs to be looked at before further interpretation is made. This task would certainly need to be undertaken by someone knowing the scientific question and the background to the data-set, information which we do not have.

Separation is a topic which has been handed elsewhere on this site and fortunately there is an excellent answer in this Q&A How to deal with perfect separation in logistic regression? (in my opinion the highest voted answer, not the accepted one is the one to go for if you are short of time to read them all).

you can not delete categories randomly- you need to investigate further as which two variables are highly correlated or inserting duplicate information to your model.. then you can proceed with removing the variable that has the highest correlations with all the other variables...

dimension reduction methods such as PCA can be other options..but it all depends on the type of data you are dealing with

Since these categories are coming from single variable, just combine those highly correlated levels into single one. say there are levels 'A' and 'B'. Then just create 3rd variable if 'A' or 'B' then 'C'. Now remove 'A' and 'B' and use 'C' in your model.

• I am introducing the ordinal variable (5-Likert) as a covariate in the ordinal model, as if I do it as factor, I have a warning of quasi-complete separation and I do not know how to deal with it. Do you think is better let the variable in the model even with the multicollinearity or maybe change it to a 3 categories (dislike, neither, like)? When I introduce it as a factor I do not get the same results as a covariate. Aug 30 '16 at 11:05
• Can you please post some results, in order for me to suggest you better. As for quasi-separation is considered, I think there might be some class(es) which are present only in one of the categories of target variable.
– muni
Aug 30 '16 at 12:19
• I have a independent variable 5-point Likert item, from strongly dislike to strongly like. The categories of 'neither like or dislike' and 'like', when recoding the variable in 4 categories to study the collinearity (VF>10). If I ignore it, and I introduce the variable in the model as continuous, I have it is not significant, and a few variables are. If I introduce it as a continuous, it is significant, but some others that before were significant, are not anymore. Aug 31 '16 at 9:12
• I have another videogame, another file of data, with the same dependent variables (likelihood to buy the videogame). The results are different as the videogame is also different. I did not have problems of multicollinearity, so I run the model with the ordinal variables as continuous. The problem if I introduce the ordinal variables as factor is that I have a warning: Aug 31 '16 at 9:15
• There are 960 (85,7%) cells (i.e., dependent variable levels by observed combinations of predictor variable values) with zero frequencies. Unexpected singularities in the Fisher Information matrix are encountered. There may be a quasi-complete separation in the data. Some parameter estimates will tend to infinity. The PLUM procedure continues despite the above warning(s). Subsequent results shown are based on the last iteration. Validity of the model fit is uncertain. Aug 31 '16 at 9:15