I have a classification problem with only 2 classes, about 100 features and thousands of observations, each belonging to either of the two classes.

Currently I´m doing a PCA prior to machine learning algorithms, which is very successful indeed, so I was wondering whether another dimensionality reduction may be even better than PCA.

What and how could I do dimensionality reduction in a different way than PCA?

I already thought about LDA, but as I have 2 classes only 1 dimension would remain, which would be way too little for correct class prediction on test observations.

Any ideas?

  • $\begingroup$ auto-encoders are more powerful, but have their own caveats: stats.stackexchange.com/questions/261265/… $\endgroup$
    – dk14
    Feb 22, 2017 at 16:17
  • $\begingroup$ What are you using the pca to accomplish? What need does the dimensionality reduction serve in the context of your regression. $\endgroup$ Feb 22, 2017 at 16:56

1 Answer 1


You have many options. You could check the correlation between features and remove features which are highly correlated with other ones. You could build a random forest with the data and observe the feature importances that result from this, and remove the ones which have low importance. You could do something similar with logistic regression to take a subset of the features as well. Here is a nice discussion of that. Importance of variables in logistic regression


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.