0
$\begingroup$

I am using LIBSVM for classification and the default RBF Kernel. To get optimal parameters for C and gamma, I am using the grid.py tool included in the LIBSVM package. It is based on a k-fold cross-validation, as far as I know. I have a training and a testing set of 15 451 instances with 5 features.

Should I run the cross-validation on the whole training set or split it into e.g. a half and to the validation on it. And is there a rule for how many folds k I should chose in respect to the size of the cross-validation sample?

For example, I wanted to do the cross-validation on the whole training set (15451 instances) with k = 50. Is this a good approach for getting the best parameter values?

$\endgroup$

migrated from stackoverflow.com May 1 '14 at 18:09

This question came from our site for professional and enthusiast programmers.

1
$\begingroup$

The optimal amount of folds depends on several things, particularly the amount of instances you have and the stability of the classification algorithm. In your case many training instances are available so there is no practical limitation in number of folds there. SVMs are quite stable, which means that the classifiers won't differ very much across different folds. This means you don't need to use many folds.

That said, 10-fold cross-validation is commonly used. Using 50 folds is definitely overkill in your situation.

If you have the time for it, repeated cross-validation may be interesting (e.g. 3x 10-fold cross-validation), though some researchers advocate against it.

$\endgroup$
  • $\begingroup$ @user3585509 if you like Marc's answer then upvote or accept it $\endgroup$ – Andrew Cassidy May 7 '14 at 17:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.