I an using scikit-learn as a toolset.

I have 1K features as candidates and am trying to reduce the feature set as I believe the majority is noise (but am not sure).

I wanted to somehow automate this using PCA and Random Forests.

My end result would be a designated feature set.

Any suggestions? I know random forests provides feature_importances that can be matched to feature_names.

I know PCA can looks at features in terms of percentile that account for varience.

Before I dive in the shark tank, any advice would be greatly appreciated. Thanks, Chris

  • 1
    $\begingroup$ Random Forests thrive in high-dimensional setting. Not sure why you would want to go down the route of PCA - which by the way does not take into account the target (Y) you are trying to predict. I would recommend using the in-built feature importance provided by random forests to select a smaller subset of input features. Just be mindful that random forests have a tendency to bias towards variables that have more no. of distinct values (i.e., favour numeric variables over binary/categorical values) $\endgroup$ – user12555 Jan 21 '14 at 13:57
  • $\begingroup$ Thanks for the advice. I was not aware no. of distinct values (i.e., favor numeric variables over binary/categorical values). If you could provide me further pointers on this bis I would appreciate it) Also any information on when and when not to use SVMs vs Random Forests vs Neural Networks is appreciated!!! $\endgroup$ – Chris Rigano Jan 31 '14 at 3:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.