Plot for unbiasedness

Question

I was reading a paper and the authors describe a plot which they used to determine whether their estimator was unbiased. This plot is described as follows (verbatim):

To assess if the probabilities of taxonomic placement were unbiased, we ordered the classification probabilities from lowest to highest, and computed a cumulative sum of both these probabilities as well as the indicator variables describing whether the outcome predicted with highest probability was a correct one. We then plotted these two cumulative sums against each other. If the classification probabilities are unbiased, such a plot should follow the identity line.

Is there a specific name for this type of plot? Is there a reference which describes the theory justifying this plot?

Source:
Somervuo, P., et al. "Quantifying uncertainty of taxonomic placement in DNA barcoding and metabarcoding." Methods in Ecology and Evolution (2017): 398-407.

Answer 1

This sounds like a qq-plot to me. It's most typically used to compare the distributions of two samples (e.g., groups), or to compare a sample to a theoretical distribution (e.g., the normal). (It may help you to read my answer to: PP-plots vs. QQ-plots.) I gather they have ordered categorical distributions, and they are plotting the ECDFs (empirical cumulative distribution function) against each other. If I'm not misunderstanding, what they did seems pretty reasonable to me, although using an agreement chart might be better.