# Difference among bias, systematic bias, and systematic error?

Is there any difference among the following terms or they are same?

1. Bias
2. Systematic bias
3. Systematic errors

If there exist some differences then, please explain them. Can these errors be reduced when one increase the sample size?

UPDATE: My field of interest is statistical inference. I mean to say that how we differentiate these term as a statistician.

• It would be useful to indicate what field of study you are interested in. It is clear from the replies already offered, for instance, that "bias" has specialized meanings that differ from that of statistical analysis (in the theory of estimation, bias is the difference between the expectation of an estimator and the value of its estimand). Your question is now tagged with "epidemiology" because the replies currently come from that field, but that might or might not be what you're really interested in. – whuber Nov 25 '11 at 22:05
• As I understand, in statistics bias is the difference between estimator and estimand, where in epidemiology, bias is the non-random difference between estimator and estimand. When I see terms like 'bias' and 'systematic error' in the context of biostatistics, I tend to think of the epidemiologcial interpretation. But then again, as a student of epidemiology, I'm biased. This set of slides from Sander Greenland touches on both concepts, but focuses on epidemiology. – jthetzel Nov 26 '11 at 1:38

The term "bias" appears in two ways in the fundamental literature on statistics:

1. "...the bias $\mathbb{E}_\theta[\delta(X)] - g(\theta)$, sometimes called the systematic error, ..." [E. L. Lehmann, Theory of Point Estimation, 1983. This is a classic text.] In Lehmann's notation, which is standard, $\mathbb{E}_\theta$ is the expectation when the distribution is given by the parameter $\theta$, $\delta$ is an estimator, $X$ is an observation, and $g(\theta)$ is a property of the distribution to be estimated (the estimand). In other words, the observation (or sequence thereof) is a random variable, which makes the estimate random, and the bias is the expected deviation between the estimate and the estimand. It depends on the (unknown but true) distribution $\theta$, making it a function of the true distribution. Lehmann devotes an entire chapter to unbiased estimators: those with zero bias regardless of the value of $\theta$.

2. In measurement theory, "bias" (or "systematic error") is a difference between the expectation of a measurement and the true underlying value. Bias can result from calibration errors or instrumental drift, for example. Contrast this usage with the previous: here, a bias is a property of a measurement, which is a physical process, whereas before it was a property of a statistical estimator (which is a mathematically defined procedure to make guesses from data).

"Systematic bias" appears to be used only when distinguishing bias from random "error": the term "error" tends to be used primarily for random terms with zero expectation.

In many cases, bias in the first sense decreases as the amount of data increases: many biased estimators in practice become less and less biased with more data (although this is not theoretically guaranteed, because the concept of bias is so broad). A good example is the maximum likelihood estimator of the variance of a distribution when $n$ independent draws $x_i$ from that distribution are available. The ML estimator is

$$\hat{v} = \frac{1}{n}\sum_{i=1}^n (x_i - \bar{x})^2,$$

for $\bar{x} = \frac{1}{n}\sum_{i=1}^n x_i$. It is well known that this is biased; the estimator $\frac{n}{n-1}\hat{v}$ is unbiased. Whence, as $n\to\infty$, $\hat{v}\to\frac{n}{n-1}\hat{v}$ becomes asymptotically unbiased.

Bias in the measurement context (the second sense), however, is usually not reducible by taking more measurements: the bias is inherent in the measurement procedure itself. One has to estimate and reduce the bias by calibrating the measurement procedure or comparing it to other procedures known to have no (or less) bias, estimating the bias, and compensating for that.

This brief description of the terminology as it is used for statistical inference does not supplant the extended and more specialized replies already posted. Instead, it is intended to serve as an introduction to them and as a mild warning to be wary of universal generalizations made in limited contexts, such as "all three [terms] are equivalent to 'systematic error'," which clearly can be correct only in a narrow sense, because the two definitions I have quoted are not equivalent. Reading the other replies has alerted me to the possibility that the literature in specialized fields like epidemiology may be using familiar, standard statistical terms like "bias" in unexpected ways, some of which may actually contradict statistical definitions. In the end, in any particular situation we need to look for a clear definition that is appropriate for the context.

• Thank you for an interesting post. I suspect that a lot of epidemiologists "borrow" terms from statisticians, adapt them to their setting and then... due to the lack of a sound base... it becomes somewhat of a mess – Max Gordon Nov 26 '11 at 8:54
• @Max Gordon: whuber provides a great answer. Perhaps other fields have not haphazardly borrowed terms from mathematical statistics, but defined terms to suit the objectives of the field. Regardless, it is helpful to statisticians, epidemiologists, and others to be aware of these subtle but significant differences in terminology, especially given the prevalence of interdisciplinary collaborative efforts. – jthetzel Nov 26 '11 at 17:43
• +1 great answer! Very clear, and still rigorous and concise. – gung - Reinstate Monica Nov 27 '11 at 2:56
• @jthtzel, the mathematical statistics need not be interpreted as something that has several meanings. It is an application-led science implying that it takes care of problems associated with measurement(statistics) and truth i.e. mathematics. As of now, the literature suggests that measurement error behaves randomly and therefore, a statistic(mean and variance) remains unaffected. And in case of a mean, so-called constant bias could cause it up or low than the true estimate. But, variance and SD remain unaffected. – Subhash C. Davar Oct 17 '13 at 15:14

If I’ve learnt anything through my epidemiology studies is that this is a mine-field where there is no true right or wrong. I like statistics because it at least has a fundament in math while epidemiology is more opinion. That said I’ll try to answer your question.

From M. Porta A Dictionary of Epidemiology 5th ed. there is no mentioning of systematic bias and systematic error says “See BIAS”. This leaves bias that is described as: “Systematic deviation of results or inferences from truth. …leading to results or conclusions that are systematically (opposed to randomly) different from the truth.” I would say that there is no unsystematic bias since they all deviate your results away from the true risk estimate. The most important thing about bias is that you can’t reduce it by increasing sample size.

There are many types of bias, I’ve heard that one of the original articles on bias contained over 300 different types. The important thing is to identify them before you start your study and then try to set up your study/experiment to avoid bias. In epidemiological studies it is very useful to separate bias into three categories:

• Selection bias
• Information bias
• Confounding

Selection bias is when you select the wrong type of individuals for your study. Let’s say you’re interested in seeing if working in a coal mine is a risk – if you look for your study individuals at the coal mine you might find that they’re healthier than the general population just because the fact that the ones that are sick are no longer working at the coal mine i.e. you select the healthiest individuals and your no longer studying the source population but a subsample. Selection bias is usually the most malignant type of bias because it’s so hard to identify.

Information bias is when your data collection concerning outcome or exposure is faulty. A common error is the surgeon that asks his patient if he’s better after the surgery. Here both the patient might not want to disappoint the surgeon and reports a better outcome that he/she would otherwise and the surgeon might not want to admit that the surgery was a failure, reporting and interviewer bias.

Information bias is also known as observational bias. When it is an error in a continuous variable it’s a measurement error while in the setting of classification you have misclassification bias. Misclassification means that a study individual can end up in the wrong category, a smoker can be misclassified as a non-smoker either by chance or by reporting bias. Even if the misclassification is by chance (non-differential misclassification) it will still tend to underestimate the risk in a systematic way, especially when you have few categories. Although an excellent study by Jurek et al. 2005 showed that you should be careful making this assumption based on a single study. In regard to your question I might imagine that this is the “non-systematic bias” that the systematic bias relates.

Confounding is factors that are associated with both the exposure and the outcome and relates mor closely to the study individual. For instance Lambe et al. 2006 showed that smoking during pregnancy increases the risk for low school performance but when looking at siblings in a subpopulation where the mother had stopped smoking during her second pregnancy their school performance was just as bad. This suggests that smoking is not the cause for bad school performance but perhaps a confounder for other social factors.

This article by Sica et al. 2006 goes into more detail. What you have to be prepared for is that there really is a lack of consensus in the field for the terminology. My dream is that WHO one day produces a list of definitions that is easy to understand, makes intuitive sense and where the debate finally may end.

• If you say that bias never decrease then, how would you justify this definition? 'An asymptotically unbiased estimator is an estimator that is unbiased as the sample size tends to infinity.' – Biostat Nov 25 '11 at 21:03
• I agree with @jthetzel, I'm not sure that I quite understand your question. An unbiased estimate is one where there is no bias and where you can rely on your sample size, lean back and let the statistics do the work (...and yes, it probably never occurs in reality where you always have some type of bias). I try to keep it simple: bias is to me always a systematic error but as I said - there are far to many books on this issue and unfortunately many are written by people who barely grasp statistics. Ask an epidemiologist about effect modification - many (most?) think of it as a kind of magic... – Max Gordon Nov 25 '11 at 21:48

Terminologies may vary from from field to field. However, using terms defined in the comments below:

Is there any difference among the following terms or they are same?

No, all three are equivalent to 'systematic error'.

Can these errors be reduced when one increase the sample size?

No, increasing sample size reduces random error, not systematic error.

Comment

These terms are taken from the field of epidemiology, specifically from Rothman and colleagues discussion of error in chapters 9 and 10 of Modern Epidemiology.

To summarize:

The goal of an investigator is to provide an accurate estimate of some measure (e.g. mean, relative risk, hazard ratio, et cetera) within a population. An accurate estimate is one that is both valid and precise. A valid estimate will have a point estimate (eg. mean, relative risk, hazard ratio, et cetera) that is close to the true value in the population. A precise estimate will have narrow confidence levels around the point estimate. In addition, an estimate can be internally-valid, relative to the study population, and externally-valid, relative to a generalized population.

Departures from accuracy are caused by error. There are two main types of error: systemic error and random error.

Systemic error, often referred to as bias, results in estimates that are not valid. Systemic error includes error due to confounding, selection bias, and information bias. Confounding can generally be corrected for with techniques such as stratification or regression. Selection and information bias have traditionally been either ignored or only qualitatively assessed in analyses, probably due to unfamiliarity with appropriate bias analyses. However, methodologies for qunatitative bias analysis do exist (e.g. Lash TL and AK Fink (2003)).

Random error results in estimates that are not precise. Random error includes sampling error and random measurement error, among others. Methods to increase precision include increaseing study size, increasing study efficiency, and precision-optimizing statistical analyses such as pooling and regression.

Update

To illustrate why increasing sample size does not decrease systematic error with the dartboard analogy (copied from this CV post):

No matter how many darts are thrown at the board, the point estimate is not going to shift towards the true bulls-eye when there is 'high bias'. Here 'bias' is equivalent to 'systematic error', and 'variance' is equivalent to 'random error'.

• If you say that bias never decrease then, how would you justify this definition? 'An asymptotically unbiased estimator is an estimator that is unbiased as the sample size tends to infinity.' – Biostat Nov 25 '11 at 21:09
• @biostat: An unbiased estimator contains no systematic error, but could contain random error. Thus as the sample size increases, variance decreases, and the estimator converges on the true value of the parameter in the population. A biased estimator would contain systematic error and would not converge on the true value of the parameter in the population (unless multiple biases in the estimator happened to cancel each other out). – jthetzel Nov 25 '11 at 21:26
• @biostat: Perhaps another way to think of it: 1) An asymptotically biased estimator's probability distribution might include the true value at small sample sizes, among other values, but will converge on a value other than the true value as sample size tends to infinity. 2) An asymptotically unbiased estimator's probability distribution might include the true value at small sample sizes, among other values, but will converge on the true value as sample size tends to infinity. – jthetzel Nov 25 '11 at 22:30
• Then Bias and Systematic Error are not same? because here bias can have random error as you said? What would you think? – Biostat Nov 25 '11 at 22:49
• @biostat: As stated above, terminologies may vary from field to field. I defined bias as systematic error. You seem to be defining bias as error. In epidemiology, bias is systematic error, at least to those who follow the terminology of Rothman's canonical textbook. Perhaps you can add context to your original question to steer the responses in the appropriate direction. – jthetzel Nov 25 '11 at 23:08

These power point excerpts have some info to supplement what jthetzel and Max Gordon have given. They're oriented toward survey data, and they're not rigorous or formal, but then if you wanted that type of answer you'd probably be looking in textbooks on measurement theory or survey methods.

• While we're at it, here's another set of slides from Greenland. – jthetzel Nov 26 '11 at 1:21