# What is the difference between sampling error and measurement error?

The terms sampling error and measurement error in statistics field are very relevant and have a paramount importance for data analysis and reporting.

• This question or the discussion about it sounds familiar. I could find a related post stats.stackexchange.com/questions/403185 but I thought there was also a more closer previous question. Jan 29 at 23:09
• There are several posts closely related stats.stackexchange.com/… Jan 29 at 23:13

Suppose you are doing a study where you want to determine the distribution of the length of frogs in a particular area of wetlands. Sampling error occurs because you only caught eighty frogs and the population of all the frogs in the wetland is much bigger than this. Measurement error occurs because one of your research assistants measured the lengths of some of the frogs you caught incorrectly (say, without measuring from the correct body part when fully stretched out). More measurement error occurs because another one of your research assistants messed up and listed one of the frogs as being 8.6m in length instead of 8.6cm.

• You do know that research assistants take and record the measurements rather than professors which increases the data quality ;).
– JimB
Jan 29 at 23:14
• @JimB There could be some truth in that.💀 I've seen all sorts.🤣 Jan 29 at 23:16

Adding to @LevG's answer, I think it's useful to point out that statisticians use the term "measurement error" much more broadly than the people who actually make measurements do.

Consider blood tests for LDL cholesterol. A biochemist would use "measurement error" to refer to the difference between the total cholesterol and HDL cholesterol concentration and triglyceride concentration in the blood sample and the numbers that get reported. They would not include the error in the formula (the "Friedewald equation") for estimating LDL cholesterol from those measured quantities, nor would they include the extent to which the cholesterol concentration in the blood sample was different from your long-term average cholesterol because of variation from hour to hour and day to day.

Or consider particulate air pollution. A chemist or engineer would use "measurement error" to refer to the difference between the actual mass concentration of particles in the air at the measurement site and the number reported. They would not include the difference between the concentration at the measurement site and the concentration near you (personal exposure), nor the extent to which some of the particles get filtered out by your nose and mouth (personal dose)

Statisticians would often call all these differences "measurement error"

Sampling error can be a confusing term (indeed, it was my case when I first encountered it). When you take a sample from a population, it is likely that the mean of this sample will deviate from the true mean of the original population (although, by the law of large numbers, the larger the sample, the closer our value is to the true mean). If you take enough samples, the sample means will be distributed following a normal distribution around the true mean (this is given by the central limit theorem). The standard deviation of this distribution of sample means (also called sampling distribution) is what we call sampling error.

On the other side, measurement error has to do (quite trivially) with the measuring process. This has two parts: random and systematic. While the random part has to do with fluctuations in the measures the measurement system generates, the systematic part has to do with errors about how we measure and are usually proportional to the true value e.g. a bad calibration of the system.

As you can see, they have totally different origins.

First you have to think about what Statistics calls "Error".

Error is any difference between what you want to know and what you know. If the "actual" thing you want to know has value X, and your process produces the value Y, the total Error is just Y-X - their difference.

We then break down the ways in which what our process produces can differ from what we actually want to know. Each component of what can cause a difference is also called an Error, and the total Error is just their sum. In some cases, the Errors can have opposite signs and cancel each other out.

Error doesn't mean "mistake", it is anything that (possibly) contributes towards the value we get not being the value we want. (I say possibly, because sometimes a given Error (by hook or by crook) is zero).

To illustrate the difference between Sampling Error and Measurement Error, we can look at what it means for one of those Errors to be eliminated.

Suppose we have a bag of marbles. Some are blue, some are red. And, as a Statistician, we are obsessed about marbles in bags.

Thus, we want to know what portion of marbles in the bag are each colour. This has a true value, which we don't know, and we want to at least have a good idea what it is.

To help us, we have a device that, when it looks at a Marble, can tell if it is red or blue. This device is often called a research assistant.

If that device is imperfect and sometimes says the wrong colour, that is a Measurement Error. If it got the answer wrong 10% of the time (randomly, but consistently for each marble), and we looked at every single marble in the bag, our result would still have a 10% Measurement Error. It would have a 0% Sampling Error.

On the other hand, if that device was perfect, and we sampled 100 of the marbles at random, the result we get here would have no Measurement Error, but would have a Sampling Error, because while the bag has 1 million blue and 1 million red marbles, our sample might have 52 red and 48 blue - not the exact same proportion as the actual marbles. The Sampling Error doesn't mean our study has a mistake, it is just a way for the value we calculate to differ from the actual value.

If we sample 100 of the marbles randomly randomly, you can calculate a sampling error of this sample and understand how likely it is for your sample to be close to the actual proportion of red and blue marbles in the bag. (It will be less than +/-10% here - 1/sqrt(sample size) - so long as there is at least a modestly large number of marbles as a quick upper bound. You can do fancier math to get a tighter bound.)

You can also have systematic Sampling Errors. Suppose your random selection of balls isn't actually random - it is more likely to pick balls near the top of the bag - and the bag's distribution of ball colours isn't uniform either. Or they took balls from the wrong bag. Or the bag contained gravel as well, and the researcher took data that mixed gravel and ball samples. Or, some of the balls slipped out of the sampler's hands when randomly selected (and it turns out that one colour of ball is more likely to be slippery).

The errors here are all which balls I'm gathering additional information about. If the balls I'm gathering additional information about don't reflect the population I'm trying to find out information on, we have a Sampling Error.

Measurement Errors, meanwhile, have to do with the gathering of information from each ball. If that measurement doesn't actually produce the information I want, then it contributes to Measurement Error. I might really want to know what percentage of balls are poisonous, and I believe that the red balls are poisonous, so I measure how many balls are red - however, it turns out only 90% of the red balls are poisonous, and 10% of the blue balls are poisonous. What I measured from the samples did not match what I wanted to know.

Breaking them down like this is helpful, because if your goal is to understand how much error your results have understanding the sources of error can help you measure, mitigate, and describe them in your final results.

For example, if you have a higher reliability means of determining ball colour than your research assistant device, you could measure the research assistant against the higher reliability one to determine a value for your Measurement Error.

Similarly, if you are measuring a proxy value like colour but really want to know toxicity, you can take a sample of balls of each colour and use a series of research assistants to determine how many of each kind are poisonous.