0
$\begingroup$

I'm not sure if I can consider three different land cover maps "classifiers" but I have several pairs of TPRs and FPRs for these 3 land cover maps which look like this.

For A(corine):

 cities        TPR           FPR

Bristol     0.6894999    0.08716076    
Brussel     0.8065292    0.18621056   
Amsterdam   0.8234692    0.07085285   
Berlin      0.6944172    0.05507682  
Manchester  0.6360882    0.11037915  
...........

For B(globcover):

 cities      TPR         FPR

Bristol    0.3830600 0.02031096  
Brussel    0.7203415 0.09221805    
Amsterdam  0.5984948 0.03149400    
Berlin     0.3902973 0.01440764     
Manchester 0.7105581 0.02429963
..........

For C(grump):

 cities      TPR         FPR

Bristol    0.3830600 0.02031096  
Brussel    0.7203415 0.09221805    
Amsterdam  0.5984948 0.03149400    
Berlin     0.3902973 0.01440764     
Manchester 0.7105581 0.02429963
..........

with these datas, I plotted FPR vs TPR with a random classifier without having any threshold. And the plot looks like this:enter image description here

Can it still be considered an ROC Curve or am I missing something or have I not understood what an roc curve is?

Any help is much appreciated.

Thanks!

$\endgroup$
  • $\begingroup$ It looks like for any given city and classifier you can get different values of (TPR, FPR) by varying the classifier threshold. That is, each combination of classifier and city should have its own ROC curve. $\endgroup$ – James Sep 11 '15 at 17:00
  • $\begingroup$ yes, so i was wondering if the above plot is correct. @James $\endgroup$ – tropicalbath Sep 16 '15 at 18:22
3
$\begingroup$

No, it is not correct because each combination of classifier and city should have its own ROC curve that is obtained by changing the classifier's threshold. What you can do is compute ROC area for each (city, classifier) combination and then average by city to compare classifiers. You could also plot those ROC values using three boxplots, one for each classifier.

$\endgroup$
  • $\begingroup$ Ah, thank you! But these TPRs and FPRs that I got for each city doesn't have any threshold and to draw an roc curve for each city and classifier combination I would have to have different tprs and fprs for each city varying the thresholds. And if I choose classifiers' threshold randomly, how do i get the different tprs and fprs for each city? Or, to compare the classifiers, is it okay to just calculate accuracy rate of each city using tp,fp,fn n tn and plot city vs accuracy. Would that be the correct way to compare the accuracy of classifiers without roc curve? @James $\endgroup$ – tropicalbath Sep 17 '15 at 6:34
  • 1
    $\begingroup$ How come you can't specify a threshold for a classifier? $\endgroup$ – James Sep 17 '15 at 22:16

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.