# Good metric / method to evaluate balanced multiclass classification when some classes are more similar than others?

Surely I'm not the first person trying to do this, but can't find a good answer (probably because I'm not searching with the right terms).

I have a problem with 10 balanced classes (0-9) where the error is less important the closer the predicted class is to the actual class (e.g. if true class is 2, I'd rather classify it as 3 than as 8). Essentially, I'd like the confussion matrix to be as centered on the diagonal as possible.

Is there a specific metric/technique that is used in these cases? If so, is it implemented in sklearn or similar?

Thanks!

• Is your variable ordinal like low/medium/high? (A movie-style “star rating” counts as ordinal.) – Dave Feb 10 at 11:28
• Yes it is. It's a product quality index (0 would be worst quality, 9 would be best quality). – RR_28023 Feb 10 at 11:32
• There are specific methods for doing ordinal regression, since, as you’ve noticed, ordinal variables are kind of a hybrid of numerical and categorical. – Dave Feb 10 at 11:38

rmse <- function(predicted, true) {