4
$\begingroup$

I understand that the AUC is basically the area under the ROC curve, which is the plot of the proportion of true positives versus the proportion of false positives at different probability cutoffs. However, I came across this method to estimate AUC, where they estimate AUC using cumulative events and non-events rates after binning the data. I am having trouble understanding how the latter is equivalent to the area under the ROC curve as per the former definition.

Edit: Outlining the method here in case the link doesn't work:

Calculate AUC using Cumulative Events and Non-Events

In this method, we will see how we can calculate area under curve using decile (binned) data.

  • Sort predicted probabilities in descending order. It means customer having high likelihood to buy a product should appear at top (in case of propensity model) Split or rank into 10 parts. It is similar to concept of calculating decile.
  • Calculate number of observations in each decile level. It would be almost same in each level as we divided the data in 10 equal parts.
  • Calculate number of 1s (event) in each decile level. Maximum 1s should be captured in first decile (if your model is performing fine!)
  • Calculate cumulative percent of 1s in each decile level. Last decile should have 100% as it is cumulative in nature.
  • Similar to the above step, we will calculate cumulative percent of 0s in each decile level.
  • AUC would be calculated using trapezoidal rule numeric integration formula. In this case, the x axis is cumulative % of 0s and the y axis is cumulative % of 1s.
$\endgroup$

1 Answer 1

3
$\begingroup$

The binning isn't important here, it just discretizes the problem (and thus approximates the true ROC curve). They bin according to the predicted probabilities, but it suffices to sort by them. Then at each threshold (after binning, these thresholds will just be deciles), the cumulate percent of events is just the true positive rate, and similarly the cumulative percent of non-events is the false positive rate. So those two columns actually are the points in ROC space for each threshold (row), and then applying the trapezoid rule is the usual way to compute the AUC.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.