I have around 70 features and 3000 data points divided into 6 classes. I'm applying Random Forest, Nearest Neighbours, SVM with RBF kernel and Naive Bayes. Some of the features are correlated (mostly positive).

Are there some heuristics what a good correlation threshold for removing features is (also regarding the mentioned classifiers)? Currently I'm removing features having a correlation greater than 0.9.

  • $\begingroup$ Potentially look into Variance Inflation Factors (VIF). vif in R. Correlation between factors could be misleading. $\endgroup$ – ERT Aug 6 '18 at 17:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.