# Estimating how easy it is to classify each individual observation

Is there a way of quantifying something like an AUC per observation. What I'm trying to estimate is how easy each observation is to classify. So for example, given a test set $$X$$ of size $$m$$ containing observations (rows) $${x_1, ..., x_m}$$ and a trained model $$M$$, I want to quantify how easy it is to for the model to correctly classify $$x_i$$ for each $$i \in 1...m$$ either on an absolute scale or relative to the other $$x_j$$, $$j \neq i$$. Here, $$x_i$$ is just a row containing values for a some set of features.

• This is a very crude solution, but, if it's a two-class classification, how about just getting a vector predicted class probabilities from the model and then calculating: abs(predicted_probability - 0.5)? High values will indicate the observations that the model was sure about, and low values will be the ones that were close to 50/50. If you want to include information about whether the model was right or wrong, then multiply the values by 1 if it was right and -1 if it was wrong. – Adam B. Feb 3 at 22:12