Im using rbf svm classifier with nested cross validation (5 kfold to tune hyperparameters and then leave the last 10% for testing). When tuning hyperparameters the best cv accuracy trains to around 56%, but the training accuracy is 100%! The best cv score is found for relatively high values of gamma. The test set performs significantly worse than the cross validation score (around 50%). What should I do in this case? Obviously the model is overfit, but the overfit model returns the best cross validation score. How can this be alleviated?

Note: I have ~10k data points and ~50 features so I ont think small dataset is the problem.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.