I understand that the PR-AUC provides a better accuracy estimate than the ROC-AUC in the case of highly skewed datasets. But if I have a test dataset with less than 5 positives and 10000+ negatives, can I still use it? Is there a better alternative in such cases?
Buy the way, I have calculated the ROC-AUC based on the test dataset above and I got AUC>0.75. I think this is probably an overly optimistic assessment of the classififers's performance and, hence, I am looking for a better peformance measure.