2
$\begingroup$

I have a probabilistic classifier that produces a distribution over my 3 classes - C1, C2, C3. I want to compare some new points I'm classifying to each other, to see which one is the best fit for a specific class.

For example:. for a new point X1 the classifier will output something like [0.2, 0.2, 0.6] for another new point X2 it will produce [0.2, 0.4, 0.4] so for both X1, X2 - the chosen class would be C3. Now I want to know - which of X1, X2 is a better fit to C3 I cannot simply choose the one with the highest probability for C3, because it's probability for C3 depends on it's probabilities for the other classes. X1 got 0.6 and X2 0.4, but it's possible that X2 is closer to C3 in the hyper-plane than X1, it is just less unique for C3 than X1, and therefor X1 got a higher probability.

here's a visual, in 2 dimensions:

enter image description here

X2, who got a lower probability, is clearly a better fit to the Red class then X1, which is truly unique to the Red class, but is further from the class cluster

My questions are:

how do I normalize the results of a probabilistic classifier so I can compare predictions to each other? given an output of a probabilistic classier - how can I get the actual distance from the probabilities. It must be possible because there's an exact mapping between a set of probabilities to a point in the classified hyper-plan.

Thanks a lot! Amir

$\endgroup$
3
  • $\begingroup$ I don't think this makes sense. If these were distances you could do that, but you have probabilities. Maybe try improving your classifier. $\endgroup$ Commented Sep 13, 2019 at 8:58
  • $\begingroup$ I agree there's a weird notion to it but it does sound like a common scenario. how do you compare two predictions of your classifier to each other? how do you, for example, sort the results you got for a specific class by how much they fit to that class? seems pretty common.. $\endgroup$ Commented Sep 15, 2019 at 6:40
  • $\begingroup$ This doesn't seem common to me, once you obtain the predictions, it's over. I would rather try to fix the classifier, to make better predictions. $\endgroup$ Commented Sep 15, 2019 at 19:10

1 Answer 1

0
$\begingroup$

I do not see the ambiguity in your results. For $X_1$, you get that there is a $60\%$ chance of belonging to the third category. For $X_2$, you get that there is a $40\%$ chance of belonging to the third category. The model thinks that $X_1$ is more like a member of the third category than $X_2$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.