# Difference between Bias and Error?

In statistics, what is the difference between Bias and Error?

You can say, Bias is a type of error? or Bias is an error with some tendency?

• I think they are different things. Feb 2, 2015 at 10:24
• Bias relates to expectation while error measures deviation in the sample to population. Feb 2, 2015 at 10:25
• Bias doesn't go to 0 asymptotically, Feb 2, 2015 at 11:29
• I have often seen "bias" used to describe differences from expectation that are systematic, and "error" to describe differences that are (or appear) random. Feb 2, 2015 at 14:25
• I learned that BIAS is how far off on the average the model is from the true. Feb 12, 2015 at 3:02

We can talk about the error of a single measurement, but bias is the average of errors of many repeated measurements. Bias is a statistical property of the error of a measuring technique. Sometimes the term "bias error" is used as opposed to "root-mean-square error".

• RMS error inevitably includes any bias. Feb 2, 2015 at 11:39

The term error appears in several related (but not identical) contexts throughout science in general and statistical science in particular.

Error still carries the flavour of mistake (something erroneous), at least in the context of measurement error and particularly when scientists are thinking about their data. But its primary meaning in statistical science has long since been simply that of more or less uncontrolled variation (something erratic or errant). Sampling error, for example, refers to sampling variation, the uncontrolled and uncontrollable fact that different samples, responsibly taken, will include different data; hence in general any statistics (such as means, correlations, fraction blue) based on those samples will differ from sample to sample.

In simple regression-type models, error refers to individual disturbances in specifications such as

response variable $=$ function of predictors $+$ stochastic error

and error can refer more generally to the conditional distribution of the response variables given the predictors.

Bias refers to the difference between the true or correct value of some quantity and a measurement or estimate of that quantity. In principle it cannot be calculated therefore unless that true or correct value is known, although this problem bites to varying degrees.

• In the simplest kind of problem, the true value is known (as when the centre of a target is visible and the distance of a shot from the centre can be measured; this is a common analogy) and bias is then usually calculated as the difference between the true value and the mean (or occasionally some other summary) of measurements or estimates.

• In other problems, some careful method is regarded as the state of the art and so yielding the best possible measurements, and so other methods are regarded as more or less biased according to their degree of systematic departure from the best method (in some fields termed a gold standard).

• In yet other problems, we have one or more methods all deficient to some degree and assessment of bias is then difficult or impossible. It is then tempting, or possibly even natural, to change the question and judge truth according to consistency between methods.

The two terminologies can be made consistent with the idea that systematic measurement errors have non-zero means (hence their summary quantifies bias) and random errors have zero mean. (Equivalently, that is how we label error as systematic or random.)

In mathematical statistics, standard analyses analyse whether particular estimators are biased in small samples, asymptotically, etc,, either in general or under particular circumstances.

This sketch at times implies that error is defined additively, so that

measured value $=$ true value $+$ error

but that is just the simplest situation. Nothing here rules out the idea that error may be multiplicative rather than additive, or defined on more complicated scales (e.g. in measuring proportions or percents, error may be better considered on something like a logit scale).

Comments on erroneous and erratic here were inspired by discussions in Jeffreys, Harold. 1939/1948/1961. Theory of probability. London: Oxford University Press.

The difference between the two is not only semantic, one can also express the difference in a formula: the bias-variance-tradeoff.

The following is the bias-variance decomposition as in Elements of Statistical Learning or the wikipedia page on bias-variance tradeoff:

$$\text{MSE}(\hat{\theta}) = \text{Var}(\hat{\theta}) + \text{Bias}^2(\hat{\theta},\theta).$$

Where $$\hat{\theta}$$ is the estimator for $$\theta$$, $$\text{MSE}(\hat{\theta}) = \mathbb{E}(\hat{\theta}-\theta)^2$$ is the mean square error, $$\text{Var}(\hat{\theta})=\mathbb{E}(\hat{\theta}-\mathbb{E}\hat{\theta})^2$$ is the variance of $$\hat{\theta}$$ and $$\text{Bias}^2(\hat{\theta},\theta)= (\mathbb{E}\hat{\theta} - \theta)^2$$ is the bias (systematic deviation) of the estimator.

Form this identity we can see that in the context of estimators,

• bias is an error because it is a component of the mean square error.
• not every error is a bias (unfortunately)
• (this is not related to the question) there might be biased estimators that can have a lower MSE than unbiased estimators although it is a nice property for an estimator to be unbiased.

What I present here is about the terms error and bias for estimators but I think the principles hold true for the words as they are used in statistics in general:

One can decompose error into a systematic and an unsystematic component. Bias is a name for the systematic error.

To put it succinctly, bias is the difference of the expected value of your estimate (denote as $\hat{\theta}$) with the true value of what you are estimating (denote as $\theta$).

$$E[\hat{\theta}] - \theta$$

Error is the difference of your estimate with the true value of what you are estimating.

$$\hat{\theta} - \theta$$

You can have a fantastic estimator which is unbiased, but still have error because your observed value of the estimator didn't get it exactly right.

• isn't error the absolute value of the difference between estimated and true value? Apr 17, 2020 at 8:30

Error means wrong, e.g. type 1 & 2 errors. Bias means shifted or straying from a true value, e.g. underreporting of alcohol consumption.

Sometimes error is used to refer to fundamental or unmeasured randomness, such as the error term in a regression model, or measurement error. In some cases, but not always, such error causes bias, but they are not exchangeable terms. Error will increase variance, however.

As an example, suppose families above median household income are 60% likely to vote Republican, and families below are 30% likely to vote Republican. The odds ratio is 0.6/0.5 / (0.3/0.5) or 2. However, suppose respondents on the survey misreport their income, so that 10% misclassify from low to high, and 10% misclassify from high to low - a typical problem when non-working household members respond to these surveys. The odds ratio becomes (0.60.9 + 0.30.1)/0.5 / ((0.30.9 + 0.60.1)/0.5) or 1.7

• Time to bust out 2×2 table thinking: Both estimators and measures may have small bias and have small error, may have small bias and large error, may have small error but large bias (i.e. 'precisely incorrect'), and may have both large bias and error. Mar 1 at 16:57
• @Alexis The main reason for this answer is that no answer considered Type 1 and Type 2 errors. "Error" actually does not have any rigorous definition because it's used in many senses, even in your case there are many forms of "noise as error" in modeling and measuring, and even that is only a snippet. Bias is more rigorously defined. Mar 1 at 16:59
• You got my +1. :D Say, biostatistician, did you see my recent question about the sampling distribution of RR? Wanna keep up your habit of enlightening me? <3 Mar 1 at 17:04
• I agree with you insofar as I didn’t imagine that a question about error and bias was really about Type 1 and Type 2 errors, or indeed about other types of errors in inference. Mar 2 at 8:32