I'm trying to evaluate my different feature scoring methods with the same data by cross-validation.
I used svm for both approaches, the only difference is the scoring method for each feature. e.g. feature1 scored 0.8 in method 1 and maybe 0.9 in method 2, etc. I want to know which scoring scheme is better by looking at the performance.
I did some resampling in the process and the cross-validation for multiple times for each method, so I got more than one result for each method. I calculated the mean value of AUCs under the curves.
The two approaches basically gave me same ROC curve AUCs. But the AUCs under Precision/Recall curve were different. How can I interpret this?