# Unusual approach to assess a predictive model's performance?

Context: I am working on a predictive model. Let's call it $$f$$. The outcome that $$f$$ is trying to predict is binary. It makes predictions as probabilities, i.e. for a given input $$x$$, $$f(x) \in (0,1)$$. Its performance seems to be so-so: The AUC is 0.67 and the average precision is 0.73 (compared to 58% positive cases in the test set).

I was pondering the merits of the model when I came up with this approach at assessing the model's performance that (I think/hope) takes noise and inherent uncertainty in the data into account:

I'll explain this table by explaining what the row with index (0.2, 0.3] says: Of the model's predictions on the test set, 30 predictions were between 0.2 and 0.3. Of these 30 points, 5 were positive (outcome = 1). The proportion 0.1667 is 5 divided by 30.

The idea behind this approach is frequentist in nature: If the model makes predictions of 0.25 for a number of points, then 25% of these points should have a positive outcome. Likewise for other probabilities. In this particular case the proportion does indeed lie in the given range for most of the rows. So I guess the model is actually okay?

I have tried to find this approach somewhere in the literature, but it's hard to know which keywords to use (see question 2 below).

My question is:

1. Is this a meaningful approach?

If it is a meaningful approach, then two more questions:

1. Does it have a name?
2. Is it really just the ROC curve, the precision–recall curve or even the log loss in (discrete) disguise?