I am doing K-fold cross validation and I want to plot an averaged ROC curve in MATLAB. However currently I can only plot K ROC curves in one plot but without knowing the algorithm of averaged ROC curve.

Say I have K lists of $\hat{p}$ and Y(only 0 or 1), or correspondingly I have K lists of FPR and TPR. What should be averaged? Thanks.


You should not average your k results. Instead, think of cross validation as a way to predict every instance exactly once (no overlapping test sets).

Save your predicted probabilities of each iteration, then plot one ROC curve for your k-fold CV.

  • $\begingroup$ I got your point. Actually I want "cross-validated ROC" but I might confound it with "mean ROC", so your answer is exactly what I want. By the way, the $\hat{p}$ of all instances come from K models (although they share same algorithm), is it valid enough to simply combine them in one plot? $\endgroup$ – CuteCat Oct 25 '19 at 15:19
  • $\begingroup$ Yes, that is not only valid, but also preferred. $\endgroup$ – Laksan Nathan Oct 25 '19 at 22:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.