One class of Ordinal DV values has too few observations - best way to address

Question

I am doing an ordinal classification using glmnet, with 3 level class DV: 1 (bad), 2(ok), 3(good). I am trying to fit a model to this ordinal DV, and find best features. The problem is that one of the classes, ie (1- bad), only has 50 out of 5000 observations, the other two classes have around around equal number of observations. This is leading to a number of problems, I believe, and I am trying to find the best solution to address it.

Here is what I've tried:

model <- polr(Y ~., train, Hess=TRUE)

output: 
Warning message:
In polr(Y ~ ., train, Hess = TRUE): design appears to be rank-deficient, so dropping some coefs

Also qda.fit=qda(Y~.,data=train)

Error in qda.default(x, grouping, ...): rank deficiency in group 1

So I read the following which explains there could be a number of reasons for insufficient data here: What is rank deficiency, and how to deal with it?. The main quote here below is intro. The answer suggests some causes without going into best way- at least not that I can glean - to address my specific case.

"Rank deficiency in this context says there is insufficient information contained in your data to estimate the model you desire. It stems from many origins. I'll talk here about modeling in a fairly general context, rather than explicitly logistic regression, but everything still applies to the specific context."

My thinking about options:

• Remove level 1 class and do a binomial logistic regression
• Merge the level 1 with another class somehow - not sure best way to do that
• Something else?

Option 1 is easiest but is that the best option. My concern is this: While there is very little data, there is a pattern, and it also makes sense that very few reviews would be the lowest, relative to others. I'm loath to throw the data out.

Would appreciate some insight from those more experienced.

Answer 1

Assuming your logic in using polr initially was sound, it follows from this logic that you should merge Bad and OK together and estimate the binary model. You will only lose a tiny bit of information as a result (the logic of deleting Bad is that you would expect the covariates relationship for OK/Bad to be different to Good/OK, and, if that is the case, polr is the wrong model anyway).

However, your logic may not be sound, and if so you may learn something by exploring the data to work out what is causing the error. Were it me, I would start by building CART models for OK/Bad and Good/OK. My guess is that you may find that the OK/Bad model is surprisingly good, and, by taking the time to interpret it you will learn a lot (NB: CART doesn't suffer from separability issues).