Metrics for edge detection (ODS, OIS, AP) I have a deep learning model which outputs edge maps and I would like to evaluate the accuracy of the model. Reading some papers I saw that typically the edge detection accuracy is evaluated using three standard measures: Optimal Dataset Scale (ODS), Optimal Image Scale (OIS) and average precision (AP). Also the F-measure of ODS and OIS can be computed.
However I couldn't find anywhere how they are defined and computed from predicted edge maps and ground truth maps (2D images).
 A: ODS & OIS F-scores
For a short answer, please see this link. Longer answer below:
Typically, the edge map outputs will be grayscale images, with pixel values lying between [0,255]. On the other hand, the ground truth will be binary in nature with every pixel either marked as 0 (non-edge) or 1 (edge). So how do you evaluate your output?
In this regard, your task is now to find an optimal threshold that can binarize your edge map output. Anything above that threshold (say, 128) will be marked as an edge (1), and everything below that threshold will be marked as a non-edge (0). Once you have binarized your edge maps, you will then be able to compare them with the ground truth to calculate precision, recall, and F-score.
There are two ways of computing this optimal threshold:

*

*Optimal Dataset Scale: Iterate over all possible threshold-values, and set it as a common threshold for the entire dataset. Whichever threshold gives you the best F-score for the dataset, that becomes your ODS F-score.

*Optimal Image Scale: For each image, calculate the best threshold and corresponding F-score. (Someone can correct me if I'm wrong here -) Average out all the F-scores for all images, and that becomes your OIS F-score.


Average Precision is the area under the precision-recall curve
