I know some machine learning algorithms can output the probability of predicted labels of an input sample.

For example, give a sample with three possible labels, a probability tuple (0.2,0.3,0.5) can be outputted through some probabilistic learning algorithms, such as logistic regression or probability estimate tree. Then the label with maximum probability (here 0.5) is outputted as the final prediction.

My question is, given a new sample having the predicted probability tuple (0.3,0.4,0.3), how can I quantitatively determine the confidence of that the predicted label (here the second label) is correct? (Assume the total prediction accuracy is 0.9.)

  • 1
    $\begingroup$ You have one conceptual error. The probability is the prediction, the label is a class assignment made by combining the model with some decision rule. $\endgroup$ – Matthew Drury Jun 5 '17 at 13:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.