I have a dataset that I divide into 80% for training+test and 20% for validation I have been using Recursive feature elimination for feature selection with SVM on the 80% partition. After I get the residual features I train again the SVM algorithm (with Cross Validation) and finally I use the trained algorithm on a set of data that it has never seen before (the 20% validation set) to obtain the true performance.
I do all this training/retraining to avoid the algorithm to over fit and to test if it robust (the performance should be steady when retraining the SVM on the test subset with the reduced set of features).
In the meantime, I got a question about the use of the same learning algorithm for feature selection and for actual classification. May that lead to over fitting?
While I verified that my algorithm is not over fitting, I am wondering if there is any best practice related to this. I could not find any information on this topic...maybe everyone just uses the algorithm trained during the selection process and don't do further training?!
Let me know if you have some idea about this. Thank you!
I have found this one threat with a similar question: Choice of hyper-parameters for Recursive Feature Elimination (SVM)
