I am trying to use glm(family = binomial(link = 'logit')) for a classification task with 14 classes. I know that logistic regression is used in R for binary classification and as a result it outputs the probabilities for the predicted value being either 0 or 1. But is it possible to also use it for a non-binary classification task?

I have 14 classes and 93 features in my dataset.

This is how I have written it, and of course it does not work, because this is the approach that I use when I only have two classes;

log.model <- glm(fold1$class ~ . - id, data=fold1, family = binomial(link = 'logit'))
predict.glm(log.model, newdata=fold1.test.set, type = "response")
  • $\begingroup$ You have tagged your question with multinomial-logit which is what you are looking for. Perhaps revising some of the questions and answers there might help you? $\endgroup$ – mdewey Dec 18 '16 at 14:40
  • $\begingroup$ @mdewey: well, it was me doing the re-tagging, so that people following the relevant tag can see the post ... $\endgroup$ – kjetil b halvorsen Dec 18 '16 at 15:17
  • 1
    $\begingroup$ @kjetilbhalvorsen mystery solved, I did wonder why the OP had tagged it. $\endgroup$ – mdewey Dec 18 '16 at 15:52

As you note glm won't do it: the family=binomial part, implies two-way, not multi-way.

To look through packages you already have installed, try ??multinomial and look through the results. Among others the nnet package has a multinom, and there are several Bayesian R packages that support multinomial logistic regression including brms. (You can also do searches like ??"multinomial logistic" or ??"ordinal logistic".) For packages you don't have installed, search on CRAN.

You should also specify whether the categories have any ordering or not. If there is some ordering, such as "Small", "Medium", "Large", then you will probably want to use ordinal logistic regression rather than the more general categorical logistic regression.

  • $\begingroup$ For packages you have not installed, try the sos package (on CRAN) $\endgroup$ – kjetil b halvorsen Dec 18 '16 at 15:18
  • $\begingroup$ Great answer. I just tried multinom and it worked perfectly, but the misclassification error is disastrous. equal to 82% $\endgroup$ – l.. Dec 18 '16 at 19:25
  • $\begingroup$ 14 classes and 93 features is a lot. Hope you have a lot of good-quality data and you'll probably want to do some analysis of your features. The actual algorithm/function is the smallest part of a successful model. $\endgroup$ – Wayne Dec 18 '16 at 20:32

Attempt a one-vs-all (aka one-vs-rest) system of logistic classifiers that proposes your problem as several binary classifiers. That is train multiple binary classifiers--one for each of the 14 classes. You will end up with 14 predictions. The prediction that has the largest one-vs-all is the prediction--take the maximum probability (each classifier's prediction probability ) given by each classifier for each sample as the prediction.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.