Let's suppose that I have seven classification models (RF, SVM, ANN, DT, kNN, Logistic Reg, LDA), and I would like to evaluate their performance through various thresholds. Receiver operating characteristics (ROC) curve is one of the options out there to do this evaluation. And a certain threshold (let's say the medium threshold) was adopted to tune the models and to select the salient features. Then, a classifier, such as DT (decision trees) could be with the poorest performance at this threshold but it "surprisingly" turns out to have the best ROC curve, which means it is the best over various thresholds (and it also beats SVM!). So... Do I need to tune these classifiers and conduct the feature selection at each threshold when the ROC curves are generated?