0
$\begingroup$

I am creating a graphical representation of PR results across various hyperparameter changes in an imbalanced dataset (model used was an SVM). I'm wondering if one would still consider this a "PR curve" since it's not over probability thresholds as one typically encounters with PR curves? I know it's a PR curve, but it's not the definition one usually encounters. And, moreover, what your opinion on doing hyperparameter tuning over PR curves is? (although I'm not really using the curve itself to figure out best HP's, but rather the f-scores and MCC)

Another question: Why are we using the AUC across these different thresholds as a measure of the general classifier performance and not the AUC from a grid search with HP's instead? What is so special about the probability threshold exactly? I can't find much research or definitive literature on these peculiarities.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.