I have trained my
xgboost binary classifier on a dataset which does not represent the true proportion of positive over negative observations of the population. The model has approximately 45% of positives whereas the "true" population only has 15%.
Since I have to make inference on a sample which contains the original proportion of positives/negatives (15/85), how can I end up having calibrated probabilities?
I was thinking of reducing the positive observations in my training set after the xgboost training, so to restore the original proportions, and fit an isotonic regression on it. Then, I would use this model to calibrate my output probabilities:
from sklearn.isotonic import IsotonicRegression
y_pred_train = (sampled y_pred_train)
calibr = IsotonicRegression()
y_pred_test = calibr.predict(y_pred_test)
Does this make any sense? Thanks in advance.