What is the role of correlation in prediction accuracy of Machine Learning algorithm please explain?

  • $\begingroup$ Are you referring to autocorrelation or multicollinearity? (The latter has been answered many times in the past). $\endgroup$ – Digio Sep 13 '17 at 9:10

In a parametric regression scenario where residuals are significantly autocorrelated, some issues may arise when using ordinary least squares:

  • Estimated model coefficients remain unbiased but do not possess the minimum variance property (correct values, bad confidence intervals).

  • The MSE may seriously underestimate the true variance of the errors.

  • The standard error of the regression coefficients may seriously underestimate the true standard deviation of the estimated regression coefficients.

  • Statistical intervals and inference procedures are no longer strictly applicable.

In a machine learning scenario (usually nonparametric), where statistical inference is not applicable, all but the first of the above points are unimportant. The trained model remains unbiased, which means that its predictive accuracy is not directly affected by the effects of autocorrelation. However, what does have an indirect impact on predictive accuracy is the fact that the degree of autocorrelation constitutes a proportion of the overall variation of the response.

In other words, having a high degree of autocorrelation implies that you have some "hidden" features integrated in your dependent variable and unless you extract them somehow (e.g. time series decomposition) you will not be able to regress that variable sufficiently using exogenous input alone.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.