# How to validate classification model with ordinal information

I have a Naive Bayes model that predicts 3 classes. As you increase each class it means that the condition is more severe. 0 means no condition, 1 is concern and 2 is that they have the condition. I have a model built but I am not sure how to correctly validate the accuracy.

Since 2 is further away from 0 it is less bad if a model classifies a 2 as a 1 instead of a zero. This means that my raw accuracy score is not completely correct.

Any help would be appreciated!

## 1 Answer

You could compare the inverse of the distance. Say we use the absolute value, meaning $$distance(x,y) = | x-y|$$.

The inverse of the distance from $$0$$ to $$2$$, e.g. $$\frac{1}{|2-0|} = 0.5$$, but from $$1$$ to $$2$$ is $$1$$. You just need to be careful when computing the ditance of a class to itself, setting it always to $$0$$, for example.

• That makes sense when using that to validate, would I compute the distance from the actual - predicted and use that to validate? Wouldn't that just be an error term? – Dillon Lloyd Apr 3 at 17:02
• It would just be an error if you're computing it for your training data. For a validation you simply compute it using you validation set .You can see my suggestion as a weighting for the mismatches, that gives a higher weight to the mismatches the user (you) consider more relevant. – Lucas Farias Apr 3 at 17:12