How is the ROC curve plotted in Viola Jones face detection paper?

Question

I am reading paper by Viola and Jones. There they have used ROC curve to measure the accuracy of their classifier.

https://www.cs.cmu.edu/~efros/courses/LBMV07/Papers/viola-cvpr-01.pdf

Could someone please explain how the ROC curve is plotted in case of binary classifier like face or non face? I mean how is the data points obtained.

(X,Y)= (falsepositive, correctdetection rate)

Do I have to calculate these points for every positives and negatives of my training data set. But my positive and negative data sets are of different sizes. I am bit confused.

Answer 1

All ROC are plotted the same way except the authors may choose different variable on x-axis. The idea of s ROC is to run the identification rate from zero to 100% on y-axis by changing the detection threshold. Suppose that your algorithm produces the probability p of a hit, such as logit regression. Once you got p, you need to decide whether this p is high enough to declare a hit. So you set a threshold C. If p>C then you mark it as a hit.

If you set C high enough you’ll have not too many false positives, but you’ll be missing some true positives too. ROC in the paper runs C from 0 to 1, while plotting the false positive rate on x-axis and detection rate on y-axis. When C is low you detect a lot of hits, but you also have a lot of false positives marking wrong items, so you are in the right top corner. When C is high you are in left bottom corner of s chart

Answer 2

When you use a classifier model, the model outputs probabilities that the positive result will occur. You must then set a threshold to obtain classifications. A ROC curve is the plot of the True Positive Rate on the Y-axis and the False Postive Rate on the Y axis plotted at a range of thresholds between 0 and 1.

https://en.wikipedia.org/wiki/Receiver_operating_characteristic

Here's some code I wrote in python to plot ROC curves:

(it also prints a bunch of stuff you may or may not find useful)

import matplotlib.pyplot as plt
def regressor_to_classifier(predictions, threshold = 0.5):
    output = []
    for prediction in predictions:
        if prediction > threshold: 
            output.append(1)
        else: 
            output.append(0)
    return output

def confusion_matrix(true, predictions):
    TP = 0
    FP = 0
    TN = 0
    FN = 0
    for t, p in zip(true, predictions):
        if t == 1 and p == 1: 
            TP += 1
        elif t == 0 and p == 1:
            FP += 1
        elif t == 1 and p == 0:  
            FN += 1
        else: 
            TN += 1
    print("TP = {}\nFP = {}\nTN = {}\nFN = {}".format(TP, FP, TN, FN))
    print("Precision = {}".format(str(TP / (TP + FP))))
    print("Recall = {}".format(str(TP / (FN + TP))))
    return TP, FP, TN, FN

def roc_curve(true, float_predictions):
    x = []
    y = []
    for i in range(100):
        threshold = 0.01 * i
        bool_predictions = regressor_to_classifier(float_predictions, threshold)
        print("Threshold = {}".format(threshold))
        TP, FP, TN, FN = confusion_matrix(true, bool_predictions)
        TPR = TP / (TP + FN)
        FPR = FP / (FP + TN)
        x.append(FPR)
        y.append(TPR)
    plt.plot(x, y)
    plt.xlabel("False Positive Rate")
    plt.ylabel("True Positive Rate")
    plt.title("ROC Curve")
    plt.show()