I have a model with an ordinal DV and a few IVs that are categorical (nominal and ordinal) as well as one continuous variable. I recoded all the categorical variables with 3 or more categories into dummies to run the colinearity test. I have one variable (5-point likert scale, ordinal) that showed 2 of the 4 categories with VF>10. DO you know which is the right way to proceed? Should I erase the whole variable, just one category (randomly...)? I am using SPSS.
Further information given in comments by the OP suggest that the problem here is separation or quasi-separation since 85+% of the cells formed by a complete cross-classification are zeroes.
To answer the original question posed first: the finding of collinearity in the model is not necessarily a red flag as it is sometimes treated. It may not even be an orange alert either. It is conveying important information about the data-set which needs to be looked at before further interpretation is made. This task would certainly need to be undertaken by someone knowing the scientific question and the background to the data-set, information which we do not have.
Separation is a topic which has been handed elsewhere on this site and fortunately there is an excellent answer in this Q&A How to deal with perfect separation in logistic regression? (in my opinion the highest voted answer, not the accepted one is the one to go for if you are short of time to read them all).
you can not delete categories randomly- you need to investigate further as which two variables are highly correlated or inserting duplicate information to your model.. then you can proceed with removing the variable that has the highest correlations with all the other variables...
dimension reduction methods such as PCA can be other options..but it all depends on the type of data you are dealing with