Suppose you have a very large feature set (1000s to 1000,000 features) when building a machine learning model. How do you go about selecting the features?

I know of the following methods for feature selection when the number of features is not extremely large: (1) (supervised) PCA
(2) Lasso (3) ElasticNet (4) Forward step-wise selection

  • $\begingroup$ Related to (4), step-wise VIF exclusion can be used to get rid of variables with high multicollinearity $\endgroup$ – ERT Aug 17 '18 at 17:19
  • $\begingroup$ A nice article (which you have probably read), is found here. $\endgroup$ – ERT Aug 17 '18 at 17:21
  • $\begingroup$ In the ML domain, some individuals have had success with autoencoders, which act as a way to select features from data. Several variations exist, all of which allow you to input a large number of features and output a smaller number before running all of your data through a complex neural network. $\endgroup$ – ERT Aug 17 '18 at 17:23
  • $\begingroup$ You may also want to look into some methods to reduce noise before implementing these methods (if your data are noisy, like market data). I've seen good results with Wavelet transform. $\endgroup$ – ERT Aug 17 '18 at 17:27
  • $\begingroup$ Clustering (kMeans, hierarchical) may also return results you are looking for, giving you similar results to PCA, though as we can see here, the two methods are distinct. $\endgroup$ – ERT Aug 17 '18 at 17:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.