2
$\begingroup$

I have a classification problem with only 2 classes, about 100 features and thousands of observations, each belonging to either of the two classes.

Currently I´m doing a PCA prior to machine learning algorithms, which is very successful indeed, so I was wondering whether another dimensionality reduction may be even better than PCA.

What and how could I do dimensionality reduction in a different way than PCA?

I already thought about LDA, but as I have 2 classes only 1 dimension would remain, which would be way too little for correct class prediction on test observations.

Any ideas?

Improve this question
$\endgroup$
2
  • $\begingroup$ auto-encoders are more powerful, but have their own caveats: stats.stackexchange.com/questions/261265/… $\endgroup$
    – dk14
    Commented Feb 22, 2017 at 16:17
  • $\begingroup$ What are you using the pca to accomplish? What need does the dimensionality reduction serve in the context of your regression. $\endgroup$
    – Matthew Drury
    Commented Feb 22, 2017 at 16:56

1 Answer 1

Reset to default
1
$\begingroup$

You have many options. You could check the correlation between features and remove features which are highly correlated with other ones. You could build a random forest with the data and observe the feature importances that result from this, and remove the ones which have low importance. You could do something similar with logistic regression to take a subset of the features as well. Here is a nice discussion of that. Importance of variables in logistic regression

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.