I looked at some graphs of high bias and high variance, and I don't really understand how the bias vs variance tradeoff is any different from accuracy vs precision?

  • 2
    $\begingroup$ In what ways do you think bias v. variance is the same as accuracy v. precision? You will need to walk is through your rationale. $\endgroup$ Feb 7, 2021 at 20:10
  • $\begingroup$ “Accuracy” and “precision” and used here in the sense from science, not as the terms in machine learning classification problems. And I would say that they are similar, yes. $\endgroup$
    – Dave
    Feb 7, 2021 at 20:51
  • $\begingroup$ Sample mean $\bar X$ estimates population mean $\mu.$ If $\bar X$ is not systematically above $\mu$ or below $\mu,$ then your estimation of $\mu$ by $\bar X$ is said to be 'accurate'. If sample standard deviation $S$ is small then your estimation is said to be 'precise'. Often a 95% confidence interval for $\mu$ is of the form $\bar X \pm 2S/\sqrt{n}$ with a normal sample size $n > 30.$ Good precision leads to short confidence intervals. Good accuracy leads to CIs that are correctly centered near $\mu.$ $\endgroup$
    – BruceET
    Feb 7, 2021 at 21:39
  • $\begingroup$ The ISO has recommended a use of "accuracy" that considers issues besides bias. See the Wikipedia page for example. $\endgroup$
    – EdM
    Feb 7, 2021 at 21:51

1 Answer 1


Putting aside the use of these words in the context of classification, the word "precision" is generally taken to represent "a description of random errors, a measure of statistical variability." (Wikipedia) Even "in the sense from science," though, there are two uses of the word "accuracy" distinguished on the Wikipedia page: a common use of the term and one adopted by the ISO. In the common use, "accuracy" is

a description of systematic errors, a measure of statistical bias... ISO calls this trueness.

Under the ISO usage, accuracy is:

a combination of both types of observational error above (random and systematic), so high accuracy requires both high precision and high trueness.

In that latter ISO usage, measures like mean-squared or mean absolute error could represent "accuracy."

The bias-variance tradeoff in statistical modeling is often presented in terms of the expected error when applying a model to a new test set. Quoting from ISLR, pages 33 to 34, where $\hat f(x_0)$ represents the model's prediction for an input value of $x_0$:

... the expected test MSE, for a given value $x_0$, can always be decomposed into the sum of three fundamental quantities: the variance of $\hat f(x_0)$, the squared bias of $\hat f(x_0)$ and the variance of the error terms $\epsilon$. That is, $$ E\left( y_0 − \hat f(x_0)\right)^2 = \text{Var}\left( \hat f(x_0) \right) + \left[ \text{Bias} \left( \hat f(x_0) \right) \right]^2 + \text{Var}(\epsilon)$$ Here the notation $E\left( y_0 − \hat f(x_0)\right)^2$ defines the expected test MSE, and refers to the average test MSE that we would obtain if we repeatedly estimated $f$ using a large number of training sets, and tested each at $x_0$. The overall expected test MSE can be computed by averaging $E\left( y_0 − \hat f(x_0)\right)$ over all possible values of $x_0$ in the test set.

I see "bias" and "variance" in this modeling context as somewhat different from the situation where one is describing the "accuracy" and "precision" of a scientific measuring instrument. One might roughly consider: common "accuracy" or ISO "trueness" related to "bias" in modeling, "variance" related to the inherent "precision" of the instrument, and $\text{Var}(\epsilon)$ related to inherent variability in what you're trying to measure.

Yet the nature of bias and variance in modeling differs from that in evaluating a scientific measuring instrument. In the process of modeling there is a tradeoff that needs to be made between bias and variance to improve performance on a new data set. In setting up a model, one can introduce bias in a way that decreases variance enough to lower the overall mean squared error (a measure of "accuracy" in the ISO sense). The bias-variance tradeoff is key. That's not typically the situation with designing or using a scientific measuring device.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.