Remove features with high correlation In a classification problem using Linear SVM, I am trying to remove variables which have a strong correlation (Pearson) between them from a dataset. 


*

*What is the usual threshold recommended? I currently delete variables when they have a correlation >= 1.0 or <= -1.0 but I wonder if I should use 0.5 instead. 

*Should I create my correlation matrix after or before scaling the data ?  

 A: I don't think this can be answered in the abstract, we would need to know what is the goal, and what is the variables. Some possibilities: 
Two variables which basically are different measurements of the same thing with independent errors. Then keep both (or use their mean) looks reasonable. Other cases are less clear-cut. Since you have a classification problem, the following is relevant: Feature Selection: Correlation and Redundancy  even highly correlated variables could have non-redundant information, and in the example given removing any of them would destroy its information content. But, interestingly, in that example replacing the two variables with their difference would work! That underscores that each example have to be treated on its own. Some general threshold value which would always work would be difficult to find!
A: There is no one-size-fits-all solution for this. The threshold could be judged by the researcher based on the association between the variables.
For the high correlation issue, you could basically test the collinearity of the variables to decide whether to keep or drop variables (features). You could check Farrar-Glauber test (F-G test) for multicollinearity. Here is a link to do it in R for example: https://www.r-bloggers.com/multicollinearity-in-r/
For the scaling issue, the scaling (if you mean multiplication of variables by a real number), it does not affect the linear relationship across variables. It is however recommended to scale the features for SVM if you are using Euclidean distance metric as kernel.
