I have P predictors, C classes

After performing training on a training partition I run the test partition through my classifier.

Say for a particular test instance the predicted class is Cpred with probability Ppred while the original label is Corig and the probability of it being that class is computed to be Porig by the trained classifier.

What type of inferences, insights, uses can I draw from (Ppred - Porig)?

Some specific questions?

  1. Can I use cases where Ppred - Porig >> mean(Ppred - Porig | Corig, Cpred) to look for mislabelled data?
  2. Can I use case where Ppred - Porig >> mean(Ppred - Porig | Corig, Cpred) to further partition the Classes into a new Class Corig,pred -- perhaps the original labels are not expressive enough for the underlying phenomenon?
  3. How could one interpret Pi,pred - Pi,orig == Pj,pred - Pj,orig where i and j are two instances in the test set but where Pi,pred >> Pj,pred


  1. How is Porig calculated? Both Porig and Ppred are produced by the classifier. If there are C classes -- the Classifier will output P1, ... , Pc probabilities, one for each class. The predicted class will correspond to the argmax of P1 ... Pc
  2. OK, so how is Porig different from Ppred? Ppred is the max(P1, ... Pc) while Porig is P of the labeled class that we know apriori
  3. See R code below for a simple illustration using the R


virginica_classifiedas_versicolor<-pred=="versicolor" & iris[train==2,]$Species=="virginica"
  • 1
    $\begingroup$ "the original label is Corig and the probability of it being that class is computed to be Porig" - How do you compute this probability? $\endgroup$
    – Arun Jose
    Commented May 4, 2017 at 4:45
  • $\begingroup$ @ArunJose -- as computed by the classifier in a multinomial case -- let me update the question $\endgroup$ Commented May 4, 2017 at 5:02
  • 1
    $\begingroup$ In that case how is Ppred different from Porig? If both are probabilities from the same classifier? $\endgroup$
    – Arun Jose
    Commented May 4, 2017 at 5:04
  • $\begingroup$ @ArunJose -- I also updated the question with note 2 to make that distinction $\endgroup$ Commented May 4, 2017 at 5:07
  • 1
    $\begingroup$ "Porig is P of the labeled class that we know apriori" - This is still confusing. Are you suggesting (sum of class A/ total N classes) = Porig? $\endgroup$
    – Arun Jose
    Commented May 4, 2017 at 5:10

1 Answer 1


So after looking at the edits, here's a small clue as to why Porig should not have anything to do with Ppred from the test set.

Porig, is simply the probability of occurrence of a particular record in a given dataset. Some datasets may have more of a particular class, some may have less. This says nothing about why they occur.

Ppred on the other hand is a conditional probability. For example, when predictor 1 takes a value X and predictor 2 takes a value y, in my training data, I see class A 7 times out of 10. (I'm not actually worried about how often class A occurs in isolation)

So Ppred will be exactly the same for any given unique record, irrespective of which dataset it belongs to. If it occurs once or ten times or hundred times. In each of these cases, Porig will be different, but Ppred will be the same.

  • $\begingroup$ mmm Arun I think I need to clarify my question ... $\endgroup$ Commented May 4, 2017 at 6:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.