Will correlated features reduce the accuracy of a classifier?

Most of the posts related to correlated features talk about using PCA to remove the correlation. My problem is not specific to any algorithms. Or is the question algorithm specific? I understand that correlation may cause numerical troubles in regression but what if we are talking about neural networks, etc.?

  • 3
    $\begingroup$ cause numerical troubles. After all, are you talking about correlatedness or about multicollinearity? These are different things. $\endgroup$ – ttnphns Apr 5 '17 at 10:48
  • $\begingroup$ Reduce the accuracy compared to what? $\endgroup$ – Peter Flom - Reinstate Monica Apr 5 '17 at 11:58
  • $\begingroup$ @PeterFlom to removing the correlated features $\endgroup$ – user10024395 Apr 5 '17 at 15:06

What you are referring to is called Multicollinearity. There is a lot of views on its impact on your model.

Correlation between your input variables usually cause issues as far as interpreting model coefficients.

As far as predictive accuracy is concerned it should not affect you, but where it causes the most heartache in the the interpretation of your model.

Coefficients can switch signs and be very sensitive to the data it is trained on. If interpretation is of less importance, multicollinearity is less of an issue to you.


What with some used methods independence of errors is required. This is a much weaker condition than statistical independence of features. For nonlinear models this is rarely a concern though and engineering dependent features might in many cases even improve your precision.


Not the answer you're looking for? Browse other questions tagged or ask your own question.