2
$\begingroup$

I am doing K-fold cross validation and I want to plot an averaged ROC curve in MATLAB. However currently I can only plot K ROC curves in one plot but without knowing the algorithm of averaged ROC curve.

Say I have K lists of $\hat{p}$ and Y(only 0 or 1), or correspondingly I have K lists of FPR and TPR. What should be averaged? Thanks.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

You should not average your k results. Instead, think of cross validation as a way to predict every instance exactly once (no overlapping test sets).

Save your predicted probabilities of each iteration, then plot one ROC curve for your k-fold CV.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I got your point. Actually I want "cross-validated ROC" but I might confound it with "mean ROC", so your answer is exactly what I want. By the way, the $\hat{p}$ of all instances come from K models (although they share same algorithm), is it valid enough to simply combine them in one plot? $\endgroup$
    – CuteCat
    Oct 25, 2019 at 15:19
  • $\begingroup$ Yes, that is not only valid, but also preferred. $\endgroup$
    – Laksan Nathan
    Oct 25, 2019 at 22:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.