# How to define a classification loss function for discrete ordinal values

Assume multi class classification task where we have 5 labels: 1, 2, 3, 4, 5.
For simplicity, let's assume it is the rating of movies, number of stars.

I am after a loss function which is aware of the values.
Namely for for the case $$y_i = 5$$ if the prediction is $$\hat{y}_{i} = 3$$ the loss will smaller when compared to the case $$\hat{y}_{i} = 1$$. Namely it will punish by how far the classifier was wrong.

Is there a neural network friendly loss function for a classification like this?

Is there such loss function in the context of ensemble of trees? SVM? Maybe something in Scikit Learn?

I found something at Ordinal Categorical Classification but I was wondering if there are more options? For instance, with quadratic punishment or something else.

The formula in Keras Ordinal Categorical Crossentropy Loss Function is given by: $$loss(\hat{y}, y) = (1 + w) CE(\hat{y}, y)$$

Where $$w = \frac{\left|class(\hat{y}) - class(y)\right|}{k - 1}$$, $$CE()$$ is the the cross entropy, $$\hat{y}$$ is the probabilities per class of the classifier, $$y$$ the ground truth probabilities per class and $$k$$ is the number of classes.
The operation $$class(y)$$ is basically the $$\arg \max$$ over the vector which gets the index of the class.

• I found similar question with similar answer: datascience.stackexchange.com/questions/116656/…. Dec 3, 2022 at 15:48
• This question tends to arise every now and then. I summarized the different methods for ordinal classification here, I hope you'll find it helpful. Apr 2 at 20:34

Alternatively, if you output a full probabilistic prediction with a predicted CDF of $$(\hat{F}_1, \dots, \hat{F}_5=1)$$, you can use the Continuous Ranked Probability Score (CRPS, see Gneiting & Raftery, 2007), in a discrete version: for an actual outcome $$k$$, the CRPS is
$$\text{CRPS}(\hat{F},k) = \sum_{j=1}^5\big(\hat{F}_j-1(j\geq k)\big)^2.$$
• Indeed one of the approaches I could take and tried was regression with round() operation. Yet pure classification feels more appropriate here. So I wanted to see if there is something out there. By the way, Matlab for instance allows defining the loss function as a matrix, namely we can define any loss per any pair combination. Nov 23, 2022 at 19:34