Skip to main content
Tweeted twitter.com/#!/StackStats/status/55285818184581121
edited tags; edited title
Link
chl
  • 54.3k
  • 23
  • 227
  • 388

How to get generalisation performance from nnet in R using k-fold cross-validation?

Source Link
robintw
  • 2.2k
  • 4
  • 25
  • 24

How to get generalisation performance from nnet in R using k-fold cross-validation

I'm doing some Machine Learning in R using the nnet package. I want to estimate the generalisation performance of my classifier by using k-fold cross-validation.

How should I go about doing this? Are there some built-in functions that do this for me? I've seen tune.nnet() in the e1071 package, but I'm not sure that does quite what I want.

Basically I want to do the cross-validation (split into 10 groups, train on 9, test on the other 1, repeat) and then from that obtain some sort of measure of how well my classifier generalises - but I'm not sure what measure that should be. I guess I want to look at the average of the accuracies across the different cross-validation examples, but I'm not sure how to do that with the tune.nnet() function above.

Any ideas?