0
$\begingroup$

I am doing an ordinal classification using glmnet, with 3 level class DV: 1 (bad), 2(ok), 3(good). I am trying to fit a model to this ordinal DV, and find best features. The problem is that one of the classes, ie (1- bad), only has 50 out of 5000 observations, the other two classes have around around equal number of observations. This is leading to a number of problems, I believe, and I am trying to find the best solution to address it.

Here is what I've tried:

model <- polr(Y ~., train, Hess=TRUE)

output: 
Warning message:
In polr(Y ~ ., train, Hess = TRUE): design appears to be rank-deficient, so dropping some coefs

Also qda.fit=qda(Y~.,data=train)

Error in qda.default(x, grouping, ...): rank deficiency in group 1

So I read the following which explains there could be a number of reasons for insufficient data here: What is rank deficiency, and how to deal with it?. The main quote here below is intro. The answer suggests some causes without going into best way- at least not that I can glean - to address my specific case.

"Rank deficiency in this context says there is insufficient information contained in your data to estimate the model you desire. It stems from many origins. I'll talk here about modeling in a fairly general context, rather than explicitly logistic regression, but everything still applies to the specific context."

My thinking about options:

  • Remove level 1 class and do a binomial logistic regression
  • Merge the level 1 with another class somehow - not sure best way to do that
  • Something else?

Option 1 is easiest but is that the best option. My concern is this: While there is very little data, there is a pattern, and it also makes sense that very few reviews would be the lowest, relative to others. I'm loath to throw the data out.

Would appreciate some insight from those more experienced.

$\endgroup$
  • $\begingroup$ When I create data with 50 in outcome category 1 versus ~2475 in other two outcome categories, I am still able to obtain results for the models you demo. For ordinal regression, this type of problem may be a separation problem. And the most common method for dealing with such problems is a penalization method. Usually, the regression should run, but you should obtain strange large standard errors. Check out the ordinalNet package, it is supposed to perform penalized ordinal regression. $\endgroup$ – Heteroskedastic Jim Dec 7 '17 at 4:42
-1
$\begingroup$

Assuming your logic in using polr initially was sound, it follows from this logic that you should merge Bad and OK together and estimate the binary model. You will only lose a tiny bit of information as a result (the logic of deleting Bad is that you would expect the covariates relationship for OK/Bad to be different to Good/OK, and, if that is the case, polr is the wrong model anyway).

However, your logic may not be sound, and if so you may learn something by exploring the data to work out what is causing the error. Were it me, I would start by building CART models for OK/Bad and Good/OK. My guess is that you may find that the OK/Bad model is surprisingly good, and, by taking the time to interpret it you will learn a lot (NB: CART doesn't suffer from separability issues).

$\endgroup$
  • $\begingroup$ Thanks @Tim, I'm using polr to get the t-test (and manual subsequent p-test calculation), to see which variables are significant and worth considering in an ordinal multinom classification ie using multinom method in caret with ordered=True. I'll check out CART class. tree to see what happens there and will examine the ok/bad combo. $\endgroup$ – user637251 Dec 7 '17 at 6:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.