I noticed that all feature selection methods implemented in sklearn are based on external estimator that assigns weights to features, AKA
I couldn't find an implementation of the more naive methods, of forward-stepwise selection based on scoring (actually cross-validating the model performance and recording the score) anywhere else but in an external library called mlextend
Is there a reason for that? I realize that scoring has computational cost (you should use CV to get unbiased estimation of the score), but isn't that a better approach as it actually checks performance instead of relying on a heuristic for feature importance?
Will appreciate any insight here.