I'm trying to do ROC analysis on my dataset.
Namely, I'm trying to find the optimal set of parameters for a predictive model. Given I just use discrete classifiers, all I have are points in the ROC space.
reading "An Introduction to ROC Analysis" by Fawcett (2006) tells that
"Informally, one point in ROC space is better than another if it is to the northwest (tp rate is higher, fp rate is lower, or both) of the first classifier."
So I guess my question is how to make the process "formal" with a sound methodology that I can back up with proper literature.
The wikipedia example makes it really obvious that given those 5 points, C' would be the best classifier. but that is done visually, and i would like automate the process.
Other questions based on my searches here on the stackexchange usually deal with the scoring-type classifiers (as pointed out again in Fawcett, 2006) and so I feel is not relevant to what I'm trying to accomplish.
Off of my pragmatic (and non-academic) thinking, I thought of:
using the shortest distance to the point
using the slope of the line connecting the classifier to again the point
(0,1)and using the steepest slope, supposedly to maximize TPR and minimize FPR.
but again I cannot find any literature to back these ideas.