I came across this paper: https://www.ncbi.nlm.nih.gov/pubmed/29355115
where the authors use random forests for feature selection in the following way:
"..we performed the RFE procedure 100 times with random initialization. We finally included only those features that were selected in more than 50 of the 100 repetitions and we used these to train the SVM, GP, kNN, and LR models."
I've read in several posts that feature selection done this way is supposed to be done only in the train set but because in this paper they calculate LOOCV AUC (leave-one-out cross validation) they use all the dataset to perform the feature selection. Is it correct to assume that if we use the LOO or LPO approach, we don't need to use a subset of the dataset to perform feature selection?
If not, if I use an 80-20 split for train-test, is there a way to ensure that my feature selection is not subject to the random split? Or just hope that the 80% of my dataset is representative enough to get a good subset of features?