5
$\begingroup$

Surely I'm not the first person trying to do this, but can't find a good answer (probably because I'm not searching with the right terms).

I have a problem with 10 balanced classes (0-9) where the error is less important the closer the predicted class is to the actual class (e.g. if true class is 2, I'd rather classify it as 3 than as 8). Essentially, I'd like the confussion matrix to be as centered on the diagonal as possible.

Confussion matrix

Is there a specific metric/technique that is used in these cases? If so, is it implemented in sklearn or similar?

Thanks!

Improve this question
$\endgroup$
3
  • $\begingroup$ Is your variable ordinal like low/medium/high? (A movie-style “star rating” counts as ordinal.) $\endgroup$
    – Dave
    Feb 10, 2021 at 11:28
  • $\begingroup$ Yes it is. It's a product quality index (0 would be worst quality, 9 would be best quality). $\endgroup$
    – RR_28023
    Feb 10, 2021 at 11:32
  • $\begingroup$ There are specific methods for doing ordinal regression, since, as you’ve noticed, ordinal variables are kind of a hybrid of numerical and categorical. $\endgroup$
    – Dave
    Feb 10, 2021 at 11:38

1 Answer 1

Reset to default
2
$\begingroup$

Would the root mean squared error work? You give more penalty to classes predicted far from the truth. E.g.

rmse <- function(predicted, true) {
    sqrt(sum((predicted - true)^2) / length(predicted))
}

rmse(c(1,1,2,2,3,3), c(1,1,2,2,4,4))
[1] 0.577

rmse(c(1,1,2,2,4,5), c(1,1,2,2,4,4))
[1] 0.408
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.