I looked at some graphs of high bias and high variance, and I don't really understand how the bias vs variance tradeoff is any different from accuracy vs precision?
$\begingroup$
$\endgroup$
4
-
2$\begingroup$ In what ways do you think bias v. variance is the same as accuracy v. precision? You will need to walk is through your rationale. $\endgroup$– Demetri PananosFeb 7, 2021 at 20:10
-
$\begingroup$ “Accuracy” and “precision” and used here in the sense from science, not as the terms in machine learning classification problems. And I would say that they are similar, yes. $\endgroup$– DaveFeb 7, 2021 at 20:51
-
$\begingroup$ Sample mean $\bar X$ estimates population mean $\mu.$ If $\bar X$ is not systematically above $\mu$ or below $\mu,$ then your estimation of $\mu$ by $\bar X$ is said to be 'accurate'. If sample standard deviation $S$ is small then your estimation is said to be 'precise'. Often a 95% confidence interval for $\mu$ is of the form $\bar X \pm 2S/\sqrt{n}$ with a normal sample size $n > 30.$ Good precision leads to short confidence intervals. Good accuracy leads to CIs that are correctly centered near $\mu.$ $\endgroup$– BruceETFeb 7, 2021 at 21:39
-
$\begingroup$ The ISO has recommended a use of "accuracy" that considers issues besides bias. See the Wikipedia page for example. $\endgroup$– EdMFeb 7, 2021 at 21:51
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Putting aside the use of these words in the context of classification, the word "precision" is generally taken to represent "a description of random errors, a measure of statistical variability." (Wikipedia) Even "in the sense from science," though, there are two uses of the word "accuracy" distinguished on the Wikipedia page: a common use of the term and one adopted by the ISO. In the common use, "accuracy" is
a description of systematic errors, a measure of statistical bias... ISO calls this trueness.
Under the ISO usage, accuracy is:
a combination of both types of observational error above (random and systematic), so high accuracy requires both high precision and high trueness.
In that latter ISO usage, measures like mean-squared or mean absolute error could represent "accuracy."
The bias-variance tradeoff in statistical modeling is often presented in terms of the expected error when applying a model to a new test set. Quoting from ISLR, pages 33 to 34, where $\hat f(x_0)$ represents the model's prediction for an input value of $x_0$:
... the expected test MSE, for a given value $x_0$, can always be decomposed into the sum of three fundamental quantities: the variance of $\hat f(x_0)$, the squared bias of $\hat f(x_0)$ and the variance of the error terms $\epsilon$. That is, $$ E\left( y_0 − \hat f(x_0)\right)^2 = \text{Var}\left( \hat f(x_0) \right) + \left[ \text{Bias} \left( \hat f(x_0) \right) \right]^2 + \text{Var}(\epsilon)$$ Here the notation $E\left( y_0 − \hat f(x_0)\right)^2$ defines the expected test MSE, and refers to the average test MSE that we would obtain if we repeatedly estimated $f$ using a large number of training sets, and tested each at $x_0$. The overall expected test MSE can be computed by averaging $E\left( y_0 − \hat f(x_0)\right)$ over all possible values of $x_0$ in the test set.
I see "bias" and "variance" in this modeling context as somewhat different from the situation where one is describing the "accuracy" and "precision" of a scientific measuring instrument. One might roughly consider: common "accuracy" or ISO "trueness" related to "bias" in modeling, "variance" related to the inherent "precision" of the instrument, and $\text{Var}(\epsilon)$ related to inherent variability in what you're trying to measure.
Yet the nature of bias and variance in modeling differs from that in evaluating a scientific measuring instrument. In the process of modeling there is a tradeoff that needs to be made between bias and variance to improve performance on a new data set. In setting up a model, one can introduce bias in a way that decreases variance enough to lower the overall mean squared error (a measure of "accuracy" in the ISO sense). The bias-variance tradeoff is key. That's not typically the situation with designing or using a scientific measuring device.
