I am investigating the isotonic regression approach to calibrate the scores from a classifier.

If I understand correctly, we do the following. First, we get the calibration plot (or reliability curve), which is the mean predicted values vs. fraction of positives. Then, we want the "fraction of positives" to be a non-decreasing function of "mean predicted values", which is done by isotonic regression.

Here is my confusion: how comes that in some cases "fraction of positives" is not non-decreasing function? For example, here: the calibrated case is not decreasing. The plot is taken from

https://www.svds.com/classifiers2/

One can find other examples with the same issue. I have read the original paper

B. Zadrozny and C. Elkan. Transforming classifier scores into accurate multiclass probability estimates.

In their results the calibrated function is monotone.

I am investigating the isotonic regression approach to calibrate the scores from a classifier.

If I understand correctly, we do the following. First, we get the calibration plot (or reliability curve), which is the mean predicted values vs. fraction of positives. Then, we want the "fraction of positives" to be a non-decreasing function of "mean predicted values", which is done by isotonic regression.

Here is my confusion: how comes that in some cases "fraction of positives" is not non-decreasing function? For example, here: the calibrated case is not increasing function. The plot is taken from

https://www.svds.com/classifiers2/

One can find other examples with the same issue. I have read the original paper

B. Zadrozny and C. Elkan. Transforming classifier scores into accurate multiclass probability estimates.

In their results the calibrated function is monotone.

I am investigating the isotonic regression approach to calibrate the scores from a classifier.

If I understand correctly, we do the following. First, we get the calibration plot (or reliability curse), which is the plot mean predicted values vs. fraction of positives. Then, we want the "fraction of positives" to be a non-decreasing function of "mean predicted values", which is done by isotonic regression.

Here is my confusion: how comes that in some cases "fraction of positives" is not non-decreasing function? For example, here: the calibrated case is not decreasing. The plot is taken from

https://www.svds.com/classifiers2/

One can find other examples with the same issue. I have read the original paper

B. Zadrozny and C. Elkan. Transforming classifier scores into accurate multiclass probability estimates.

In their results the calibrated function is monotone.

I am investigating the isotonic regression approach to calibrate the scores from a classifier.

If I understand correctly, we do the following. First, we get the calibration plot (or reliability curve), which is the mean predicted values vs. fraction of positives. Then, we want the "fraction of positives" to be a non-decreasing function of "mean predicted values", which is done by isotonic regression.

Here is my confusion: how comes that in some cases "fraction of positives" is not non-decreasing function? For example, here: the calibrated case is not decreasing. The plot is taken from

https://www.svds.com/classifiers2/

One can find other examples with the same issue. I have read the original paper

B. Zadrozny and C. Elkan. Transforming classifier scores into accurate multiclass probability estimates.

In their results the calibrated function is monotone.