Predicted class probability in threshold moving

Question

I am training a model for the task of Binary classification using H2O.ai. The final output to the user is the probability of class_1.

Recently, I found that by default the training uses threshold moving, and all the test_f1 are the scores found after moving the threshold.

Now, Let's say the class_1 probability is 90%, while the threshold is set at 94%. The user will be completely sure that it's a true instance while the model actually classified it as false.

My question is how can We interpret the probability of a class in a task of binary classification while threshold moving has been applied?

Thank you

Answer 1

To be meaningful as probability values, you will want them to be calibrated. That is, when a prediction of $P$ is made, the event should happen about $P$ proportion of the times. Concretely, if you keep predicting an event to happen with $90\%$ probability, yet the event happens in only $60\%$ of the cases, your prediction of $0.9$ lacks calibration.

It is not a given that your predictions will be calibrated probability values.

The two links below discuss software tools and the statistical ideas implemented in them for assessing probability calibration. I do not know of an implementation besides R, but I once ported basic functionality of val.prob to Python without much difficulty, so it can be done.

Walk through rms::calibrate for logistic regression

Walk through rms::val.prob