Let's say I have a dataset with 1000s of features. To save computation resources, I'd like to reduce this number of features.
There seems to be many ways to do feature selection.
Sklearn has some built in features: http://scikit-learn.org/stable/modules/feature_selection.html
mlxtend also has some interesting algorithms: https://rasbt.github.io/mlxtend/user_guide/feature_selection/SequentialFeatureSelector
Other than that, I find there isn't much research on best practices.
Ok, don't use RF for feature selection for linear models: Can a random forest be used for feature selection in multiple linear regression?
For machine learning, XGBoost performs well in Kaggle competitions and in many ways is a good default model.
Is there a good default feature selection technique?