Suppose you have a very large feature set (1000s to 1000,000 features) when building a machine learning model. How do you go about selecting the features?
I know of the following methods for feature selection when the number of features is not extremely large:
(1) (supervised) PCA
(2) Lasso (3) ElasticNet (4) Forward step-wise selection