Skip to main content
1 of 2
robintw
  • 2.2k
  • 4
  • 25
  • 24

How to get generalisation performance from nnet in R using k-fold cross-validation

I'm doing some Machine Learning in R using the nnet package. I want to estimate the generalisation performance of my classifier by using k-fold cross-validation.

How should I go about doing this? Are there some built-in functions that do this for me? I've seen tune.nnet() in the e1071 package, but I'm not sure that does quite what I want.

Basically I want to do the cross-validation (split into 10 groups, train on 9, test on the other 1, repeat) and then from that obtain some sort of measure of how well my classifier generalises - but I'm not sure what measure that should be. I guess I want to look at the average of the accuracies across the different cross-validation examples, but I'm not sure how to do that with the tune.nnet() function above.

Any ideas?

robintw
  • 2.2k
  • 4
  • 25
  • 24