I am building a classifier using random forest. I have separated my data to 80% training set and 20% test set.
I did the cross-validation for feature selection, and looked at the OOB error rate to choose the number of trees. I used the model I got on the test set and got the TPR, TNR, PPV and so on.
What I want is to do a k-fold cross validation on my training set to make sure I'm not overfitting, but I'm not sure which metrics should I look at. I have the AUC for each fold, should I just look at it's average and STD? Are there other parameters I could use to show how general my model is? Is it even necessary to show that?