I've seen taught several places that feature selection needs to be inside the CV training loop. Here are three examples where I have seen this:
...you must include feature selection within the inner-loop when you are using accuracy estimation methods such as cross-validation. This means that feature selection is performed on the prepared fold right before the model is trained. A mistake would be to perform feature selection first to prepare your data, then perform model selection and training on the selected features...
Here is an example from the sklearn docs, that shows how to do recursive feature elimination with regular n-fold cross validation.
However I'd like to do recursive feature elimination inside random/grid CV, so that "feature selection is performed on the prepared fold right before the model is trained (on the random/grid selected params for that fold)", so that data from other folds influence neither feature selection nor hyperparameter optimization.
Is this possible natively with sklearn methods and/or pipelines? Basically, I'm trying to find an sklearn native way to do this before I go code it from scratch.