Can one leverage the probability difference between the the predicted class vs the original class?

Question

I have P predictors, C classes

After performing training on a training partition I run the test partition through my classifier.

Say for a particular test instance the predicted class is Cpred with probability Ppred while the original label is Corig and the probability of it being that class is computed to be Porig by the trained classifier.

What type of inferences, insights, uses can I draw from (Ppred - Porig)?

Some specific questions?

1. Can I use cases where Ppred - Porig >> mean(Ppred - Porig | Corig, Cpred) to look for mislabelled data?
2. Can I use case where Ppred - Porig >> mean(Ppred - Porig | Corig, Cpred) to further partition the Classes into a new Class Corig,pred -- perhaps the original labels are not expressive enough for the underlying phenomenon?
3. How could one interpret Pi,pred - Pi,orig == Pj,pred - Pj,orig where i and j are two instances in the test set but where Pi,pred >> Pj,pred

Notes:

1. How is Porig calculated? Both Porig and Ppred are produced by the classifier. If there are C classes -- the Classifier will output P1, ... , Pc probabilities, one for each class. The predicted class will correspond to the argmax of P1 ... Pc
2. OK, so how is Porig different from Ppred? Ppred is the max(P1, ... Pc) while Porig is P of the labeled class that we know apriori
3. See R code below for a simple illustration using the R

Example:

set.seed(1)
str(iris)
train<-sample(2,nrow(iris),replace=TRUE)
rf<-randomForest(Species~.,data=iris,subset=train==1) 
p.resp<-predict(rf,newdata=iris[train==2,],type="prob")
pred<-predict(rf,newdata=iris[train==2,])
table(iris[train==2,]$Species,pred)
virginica_classifiedas_versicolor<-pred=="versicolor" & iris[train==2,]$Species=="virginica"
Porig<-p.resp[virginica_classifiedas_versicolor,"virginica"]
Ppred<-p.resp[virginica_classifiedas_versicolor,"versicolor"]

Answer 1

So after looking at the edits, here's a small clue as to why Porig should not have anything to do with Ppred from the test set.

Porig, is simply the probability of occurrence of a particular record in a given dataset. Some datasets may have more of a particular class, some may have less. This says nothing about why they occur.

Ppred on the other hand is a conditional probability. For example, when predictor 1 takes a value X and predictor 2 takes a value y, in my training data, I see class A 7 times out of 10. (I'm not actually worried about how often class A occurs in isolation)

So Ppred will be exactly the same for any given unique record, irrespective of which dataset it belongs to. If it occurs once or ten times or hundred times. In each of these cases, Porig will be different, but Ppred will be the same.