0
$\begingroup$

I have 30,000 samples with 150 features for a binary classification problem, now I plan to follow: https://stackoverflow.com/questions/23815938/recursive-feature-elimination-and-grid-search-using-scikit-learn, wherein I'll do a GRIDSEARCHCV on RFECV, wherein the data is split twice in each fold (once for GRIDSEARCH and the other for RFE). I plan to do this on the ENTIRE dataset. Now, after determining the best features and parameters, I plan to build the final model by splitting the SAME dataset (the one Ive used for gridsearch and rfe) into the usual 70:15:15 training/validation/test set and use only the best features and parameters. My reason for this methodology is that the GRIDSEARCH and RFE was a different process from building the final model, and that there is no way for the final model to know what the GRIDSEARCH + RFE has learned on the data. Is this methodology valid or correct?

If not, I plan on splitting the dataset 50:50, wherein the first 50% would be for the gridsearch + rfe process, and the remaining 50% is for building the final model using the determined best features and parameters.

Thanks

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.