I apply k fold cross validation using svms ( classification or regression, it doesn't matter) and I optimize my parameters based on the mean testing error. If I want to predict the value of one of my samples what do I do? I choose randomly one of the (perhaps best) k fold realizations where this sample happened to be on the testing set and predict its value?Or is it better if I apply the leave one out methodology in order to have one "testing phase" prediction for each sample?


Neither: you typically train a new model with the optimal hyperparameters on the entire training set and use that for prediction. Usually, it will perform marginally better than your cross-validation estimate because you are using more data for the final model.

An alternative that is sometimes used is to create an ensemble of the models you trained on each fold and then use the ensemble for predictions.

  • $\begingroup$ Yeah you are right. Thanx! $\endgroup$ – amanita kiki Nov 18 '15 at 8:10
  • $\begingroup$ One more question. If I do the ensemble thing, is it better to find the optimal parameters for each fold? Or should I stick with the optimal parameters extracted from all folds? $\endgroup$ – amanita kiki Nov 18 '15 at 8:21
  • $\begingroup$ It is best to stick with the optimal parameters you found from all folds. $\endgroup$ – Marc Claesen Nov 18 '15 at 8:25
  • $\begingroup$ Done. Thanx again $\endgroup$ – amanita kiki Nov 18 '15 at 8:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.