I have a neural network set up to predict something where the output variable is ordinal. I will describe below using three possible outputs A < B < C.
It is pretty obvious how to use a neural network to output categorical data: the output is just a softmax of the last (usually fully connected) layer, one per category, and the predicted category is the one with the largest output value (this is the default in many popular models). I have been using the same setup for ordinal values. However, in this case the outputs often don't make sense, for example the network outputs for A and C are high but B is low: this is not plausible for ordinal values.
I have one idea for this, which is to calculate loss based on comparing the outputs with 1 0 0 for A, 1 1 0 for B, and 1 1 1 for C. The exact thresholds can be tuned later using another classifier (eg Bayesian) but this seems to capture the essential idea of an ordering of inputs, without prescribing any specific interval scale.
What is the standard way of solving this problem? Is there any research or references that describe the pros and cons of different approaches?