I have around 70 features and 3000 data points divided into 6 classes. I'm applying Random Forest, Nearest Neighbours, SVM with RBF kernel and Naive Bayes. Some of the features are correlated (mostly positive).
Are there some heuristics what a good correlation threshold for removing features is (also regarding the mentioned classifiers)? Currently I'm removing features having a correlation greater than 0.9.