Surely I'm not the first person trying to do this, but can't find a good answer (probably because I'm not searching with the right terms).

I have a problem with 10 balanced classes (0-9) where the error is less important the closer the predicted class is to the actual class (e.g. if true class is 2, I'd rather classify it as 3 than as 8). Essentially, I'd like the confussion matrix to be as centered on the diagonal as possible.

Confussion matrix

Is there a specific metric/technique that is used in these cases? If so, is it implemented in sklearn or similar?


  • $\begingroup$ Is your variable ordinal like low/medium/high? (A movie-style “star rating” counts as ordinal.) $\endgroup$ – Dave Feb 10 at 11:28
  • $\begingroup$ Yes it is. It's a product quality index (0 would be worst quality, 9 would be best quality). $\endgroup$ – RR_28023 Feb 10 at 11:32
  • $\begingroup$ There are specific methods for doing ordinal regression, since, as you’ve noticed, ordinal variables are kind of a hybrid of numerical and categorical. $\endgroup$ – Dave Feb 10 at 11:38

Would the root mean squared error work? You give more penalty to classes predicted far from the truth. E.g.

rmse <- function(predicted, true) {
    sqrt(sum((predicted - true)^2) / length(predicted))

rmse(c(1,1,2,2,3,3), c(1,1,2,2,4,4))
[1] 0.577

rmse(c(1,1,2,2,4,5), c(1,1,2,2,4,4))
[1] 0.408

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.