# Relationship between High Variance in Machine Learning and Statistics

In Statistics, High Variance represents that:

Overall, elements of dataset are widely dispersed/different/spread out from the mean and from each other.

In Machine Learning, it represents:

Our parameters have over-fitted on the dataset and so we will get unnecessary curves and will not be able to predict on unseen data.

How these two concepts can be compared? Or are these independent concepts with no direct relation?

• Not quite. Both uses of high variances are statistical. Since part of machine learning is built on a statistical foundation (such as the part that talks about variance), it might have inherited one or both of these. – Richard Hardy Mar 11 '18 at 8:05

As such, we can talk of the variance of a parameter estimate, e.g., of a regression parameter. The idea is that we can draw many samples from the underlying data population, each sample of size $n$, and calculate the regression parameter estimate $\hat{\beta}$ each time. Then all these calculated $\hat{\beta}$s may have a high or a low variance. Lower variance for estimates is typically better.