I want to find the best model process for a machine learning pipeline. In other words, normalize $\rightarrow$ feature select $\rightarrow$ test model performance. For example, let's say I want to try Ridge, Lasso, and Elastic Net regression and I am doing normalization, feature selection, and a cross-validated hyperparameter search for all models. I want to pick the best out of the three.
Does it theoretically make sense to run cross-validation where I run the entire pipeline on each of the left out folds? In
SKLearn, something like this:
models = [Ridge(), Lasso(), ElasticNet()] for model in models: pipe = Pipeline([('scaling', scaler), ('feature_selection', selectorCV), ('param_searcm', gridsearchCV)]) scores.append(cross_val_score(pipe, X, y)) # get the best model pipeline from cross_val_score, fit on all my data, # and whoop there is my best model