1
$\begingroup$

I'm looking for a metric to evaluate my classification model. I have 3 differents class (0,1,2). And I want to get the average probability of the good label. For example, if my ml model get me those probas : [[0.5, 0.25, 0.25], [0.1, 0.8, 0.1], [0.7, 0.2, 0.1]] And the true labels are [0,1,0], then the evaluation result should be (0.5+0.8+0.7)/3 = 0.66.

Does this metric has a name ?

$\endgroup$
1
  • $\begingroup$ Hi Yoann, was your question answered? If so, could you please accept the correct answer with the checkmark beside it? Otherwise, what can be clarified? $\endgroup$ Apr 3, 2021 at 15:54

1 Answer 1

3
$\begingroup$

Yup! It has a name, and it makes sense to care about. It's called the expected accuracy—and alternatively, one minus the expected error rate.

The idea is that you pretend to pick a class randomly from the posterior distribution instead of picking the most probable class. (That is—stochastic decoding instead of MAP decoding.) If the probability of the correct class is 0.8, you'll still pick this 80% of the time, so you get 80% credit.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.