8
$\begingroup$

Can we use any of the various types of Deep Neural Networks for Ordinal Classification? If yes, then how? If no, then what is the limiting problem? I know that CovNets with a softmax output can be used for classification between n classes. Is there a way this can be extended/modified for ordinal classification?

Finally, I would like both class label and its ordinal output.

I think ordinal classification is part of regression (please clarify). So, I am asking for a DNN that can do both classification and regression. Is this possible?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

There are a few approaches. One is to do a one in hot encoding: https://arxiv.org/pdf/0704.1028.pdf

But there should be other approaches. In classical ordinal regression, we fit cut off values st:

  • $ P(X=1) = P(Z \leq \theta_1) = F(\theta_1)$
  • $ P(X=2) = P(\theta_1 \leq Z \leq \theta_2) = F(\theta_2) - F(\theta_1)$
  • $ P(X=3) = P(Z \geq \theta_2) = 1- F(\theta_2)$

Where Z is some latent variable and F is the CDF of the latent variable. It should be possible to directly apply this approach where Z is the second to last layer (the layer before the softmax).

Improve this answer
$\endgroup$
5
  • $\begingroup$ Note that you are (properly) answering with regard to probability estimation. The original question was falsely put in terms of 'classification'. $\endgroup$
    – Frank Harrell
    Jun 9, 2019 at 13:10
  • $\begingroup$ Once you have a probability for each class, the typical approach is then to choose the most probable class. $\endgroup$
    – Anonymous Emu
    Jun 10, 2019 at 18:37
  • $\begingroup$ Please don't do that unless there is a class that has a probability above 0.95 for all observations. If probabilities are more spread out there is too much uncertainty to ignore close calls. $\endgroup$
    – Frank Harrell
    Jun 10, 2019 at 21:26
  • $\begingroup$ @FrankHarrell Where do you get 0.95 as opposed to, say, 0.90 or 0.99? $\endgroup$
    – Dave
    Sep 17, 2022 at 0:11
  • $\begingroup$ You could say 0.99 but not so much 0.9. A probability of 0.9 means you'll be wrong 0.1 of the time if you classify it as "positive" and 0.1 may be too large. $\endgroup$
    – Frank Harrell
    Sep 17, 2022 at 15:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.