I have around 228 samples and 100 features in total. I wanted to do some stability analysis on the data, so I did repeated 10-fold cross validation on the entire dataset and obtained the mean classification scores. But the parameters of the classifiers that I have used to fit the models are all the default ones. My question is how do I tune the parameters for the purposes of doing stability analysis as above? One thing I thought was to use GridsearchCV on the entire dataset, find the optimal parameter set and then do the stability analysis. Is that fine?
Another idea was to do the tuning on the train set of every fold of the cross validation. But that way I will get the most stable set of parameters, not the optimal set.
So which one should I use?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.