I've been running some simulations using RFE on random forests and I got some puzzling results. Let's say I run RFE with CV and the highest F1 score is sometimes lower than the F1 Score I get from running RFE and then CV with the same number of folds. For instance, RFE with CV says the optimal n of features is 90 and its F1 score is 75%, whereas if I run RFE selecting only 20 features, its F1 with the same CV is 87%. Why would that happen?
On the graph shown with RFE CV F1 Scores, the greatest F1 Score would be around 65%, but RFE selecting 20 features and then CV for F1 Score would yield a greater F1 Score (e.g. 78%). Why is that happening?
Thank you in advance.