# Remove features with high correlation

In a classification problem using Linear SVM, I am trying to remove variables which have a strong correlation (Pearson) between them from a dataset.

• What is the usual threshold recommended? I currently delete variables when they have a correlation >= 1.0 or <= -1.0 but I wonder if I should use 0.5 instead.
• Should I create my correlation matrix after or before scaling the data ?
• Excuse me but how do you have feature that their correlation is greater than 1? Deleting perfectly correlated features is quite reasonable but how do you get the $>$ of your heuristic? Do you somehow calculate variances and then you normalize them? In any case regrading your second question the correlation is not relevant to the scaling of your data so that should not make any difference. – usεr11852 Jan 4 '16 at 17:41
• How are you determining which variable to remove in case of high correlation? – Jason Morgan Jan 4 '16 at 17:46
• Keep in mind that the linear correlation coefficient only detects linear correlations. Depending on what problem you're trying to solve (unstated in the post), this additional pre-processing may not even do anything helpful. – Sycorax Jan 4 '16 at 18:17
• @usεr11852 I don't have correlation greater than 1. I just used >= and <= so that I can change the number by another value and my condition still works in the code. Sorry if this was misleading. – Octoplus Jan 4 '16 at 18:56
• @user777 after some test, it looks like it does not improve my classification. However it removes a lot of variables so the model is slightly faster to train. – Octoplus Jan 4 '16 at 18:58