I'm doing some Machine Learning in R using the nnet
package. I want to estimate the generalisation performance of my classifier by using k-fold cross-validation.
How should I go about doing this? Are there some built-in functions that do this for me? I've seen tune.nnet()
in the e1071
package, but I'm not sure that does quite what I want.
Basically I want to do the cross-validation (split into 10 groups, train on 9, test on the other 1, repeat) and then from that obtain some sort of measure of how well my classifier generalises - but I'm not sure what measure that should be. I guess I want to look at the average of the accuracies across the different cross-validation examples, but I'm not sure how to do that with the tune.nnet()
function above.
Any ideas?