1
$\begingroup$

Measurement precision can be improved by measuring the same quantity multiple times and taking the average. In other words, the average of several measurements of the same thing is more likely to be closer to the true value than a single measurement. The more measurements you average, the better the precision of the average -- the average of two measurements is more precise (i.e., probably closer to the true value) than one measurement. The average of three measurements should be more precise than the average of two, and so on.

My question is, how much more precise? What is the mathematical relationship between the number of measurements averaged and the precision? In other words, how much more precise is the mean of $n$ measurements than a single measurement, where $n>1$? Given given some previously established precision metric (such as SD, CV, or 95% CI), how would one calculate the precision of measuring the same thing $n$ times and finding the mean of those measurements?

My best stab at how to do this is the apply the standard error of the mean (SEM) formula using the documented SD and the number of measurements I'm taking as the sample size. I know that the SEM formula is to divide the population SD by the square root of the sample size, and if I understand correctly that tells you the SD of all possible sample means, i.e. the precision of the sample mean relative to the true mean. If the documented SD is based on large scale testing and can be regarded as the "population" SD (i.e. the actual measurement precision) for practical purposes, then it seems to me that dividing the documented SD by the square root of the number of measurements should give me the standard error of the mean of those measurements, which is the SD of that mean relative to the true mean, which in this case is what the measurement would be with zero random error (the actual quantity being measured plus the bias).

Is that reasoning sound? If not, what's the correct approach to quantifying the uncertainty of x number of measurements, not based on the statistical spread of just those few measurements themselves, but based on the documented precision of the instrument?


Here is a concrete example to illustrate what I'm asking:

Suppose the documented SD of an electronic scale is 2.82g*. That tells me the probability distribution of a measurement with that scale. If I weigh an object once, and the result is 250g, there is a ≈68% probability that 250g is within 2.82g of the true weight, and a ≈95% probability that it's within 5.53g of the true weight (2.82 * 1.96), assuming a normal distribution.

If I weigh the same object three times, and my results are 250g, 248g, and 255g, each of those measurements by itself has the same ≈68% probability of being with 2.82g of the true weight, and ≈95% probability of being within 5.53g. However, the mean of those three measurements (251) should have a higher precision than any one measurement by itself. What would be the precision of the mean of those three measurements, expressed in terms of SD?

Note that I am not asking what is the SD of those three measurements, based on their variation from each other, or how to estimate the precision of the measurement instrument based on the sample of three measurements. I am asking what is the precision of the mean of those those measurements, expressed as SD, relative to the previously established measurement precision? If the precision in terms of SD for any one measurement by itself is 2.82g, how precise is the mean of three measurements? If one measurement has a ≈68% probability of being within 2.82g of the true value, the mean of three measurements of the same object has a ≈68% chance of being within how many grams of the true value?

If I weigh the object four times, what would be the precision of the mean of those measurements? It should have a better precision than the mean of three measurements, just as the mean of three measurements should have a better precision than the mean of two measurements, and the mean of two measurements should be more precise than one measurement, since the theoretical precision should improve the more measurements are averaged.

* Disregard any possible imprecision in the documented SD itself, or any issues that might make the precision of my scale different than the documented SD, such as calibration or drift, and assume that the precision is fixed throughout the measurement range. I'm stipulating that the documented SD is trusted to be for all intents and purposes the "population SD" for my scale, i.e. the SD that would theoretically be obtained from an infinite number of measurements of the same immutable object.

I understand that those are issues to take into account in a practical situation, but my question is theoretical, so please disregard any and all what-ifs that might affect the answer in practice. I'm asking only about the mathematical relationship between the precision of the mean of multiple measurements of the same quantity and the precision of a single measurement.

Based on my reasoning above, since the documented SD is for practical purposes being regarded as the population SD, I should be able to divide the documented SD by the square root of the number of measurements I'm taking, and that would tell me the SD of the mean of those measurements (again, not the SD of those measurements themselves, but as a means of expressing the probability distribution of their mean as an estimate of the true weight of the object).

So, for three measurements, that would be $\frac{2.82g}{\sqrt{3}} \approx 1.63g$, which would mean that the mean of three measurements would have a ≈68% probability of being within 1.63g of the true value, and a 95% CI of ±3.19. For five measurements, it would be $\frac{2.82g}{\sqrt{5}} \approx 1.26g$, and a 95% CI of ±2.47g.

Is this correct? If not, what is?

$\endgroup$
  • 1
    $\begingroup$ It might be better to use statistical terms in the usual way. For example, refer to the link to see how the terminology is usually used. $\endgroup$ – Carl Nov 17 '19 at 23:07
  • 1
    $\begingroup$ Can you please point out specifically where I have used any of those terms in any way other than what's described there? If you mean that I'm calling the documented SD the population SD, that's conceptually equivalent to what you said about "population mean" being the imaginary mean of infinite data in your Nov 19 2:08 comment below, and I'm not even saying that it is the population SD, I'm making the case that it can function as the population SD for the purpose of applying the SEM formula. $\endgroup$ – Adi Inbar Nov 23 '19 at 4:53
  • $\begingroup$ I explain these issues in my answer at stats.stackexchange.com/a/18609/919. It explicitly describes how one constructs a standard error even when the population standard deviation is unknown and gives the rationale for that: "You don't, in fact, know the standard deviation of the tickets in the hat, so you estimate it by using the standard deviation of your sample instead. When divided by the square root of the sample size, it estimates the standard error of the [mean]." $\endgroup$ – whuber Nov 23 '19 at 17:28
  • 1
    $\begingroup$ @whuber Indeed you did explain it. Now that we have a question that is rather more clear, it is a duplicate. $\endgroup$ – Carl Nov 24 '19 at 4:27
-1
$\begingroup$

Well, you need to read about small number error for SD. What is happening is that variance is unbiased, but its square root, i.e., standard deviation, is not, strange as that may seem. I asked a similar question before. Why are we using a biased and misleading standard deviation formula for $\sigma$ of a normal distribution? Also, see small number bias and similar links from that question and its answers.

Mean values thread over lists, standard deviations do not. For example, taking the mean of 2 values and doing that 5 times then taking the mean of that will return the mean of ten values. That does not work for standard deviations. The error for mean values of a normal distribution is $\dfrac{\bar{X}}{\sqrt{n}}$. That would not be true for a Cauchy distribution. The square of standard deviation, i.e., variance, threads over lists, and for a normal distribution there are two sources of error for standard deviation, the first is small number bias which consists of an underestimate of population SD, e.g., for $n=5$, the calculated SD is on average 94% of the population SD. The second is the noise, or if you will, the SD of the SD, see https://stats.stackexchange.com/a/28567/99274

$\endgroup$
  • 1
    $\begingroup$ Maybe I'm missing something, but I'm not getting why this would apply, much less how. This information is about estimating a population SD based on a small sample, but that's not what I'm trying to do. My goal isn't to derive the best possible estimate of the measurement precision of the instrument based on the variation among a small number of measurements. The precision of the instrument, in terms of SD, is provided to me by the manufacturer's documentation, and I'm regarding that for practical purposes as the true or "population" SD that quantifies the precision of any one measurement. $\endgroup$ – Adi Inbar Nov 16 '19 at 18:38
  • 1
    $\begingroup$ My goal is to estimate the precision of the mean of several measurements not based on the variation in those measurements but based on an already known "population" SD (i.e. a "good enough" estimate of the SD that would theoretically be obtained from the variation in an infinite number of measurements). The precision of the mean of repeated measurements should improve the more measurements are averaged, but with the bias correction formulas, the estimate gets higher (i.e. precision gets worse), because it approaches the population SD as the sample increases. $\endgroup$ – Adi Inbar Nov 16 '19 at 18:40
  • $\begingroup$ I'm not even sure what the bias correction formulas would apply to, since at no point would I be calculating the SD of a sample of measurements, because what I want to know is how the combined precision improves based on how many measurements, not estimate what the precision is based on any specific set of measurements ("What would the precision of the mean of any three or any five measurements be relative to the precision of one measurement?", not "What would be the precision of the mean of a specific set of three or five measurements, based on the values of those measurements?") $\endgroup$ – Adi Inbar Nov 16 '19 at 18:41
  • $\begingroup$ Given 2 values, the calculated mean is midway between them. The population mean does not even have to be inside the range between the two values, never mind at the midpoint. In general, the precision of the mean is its standard error $\dfrac{\bar{X}}{\sqrt{n}}$, where $n$ is the number of measurements, and $\bar{X}$ is the calculated mean value. $\endgroup$ – Carl Nov 16 '19 at 20:16
  • $\begingroup$ I'm more confused than ever. I'm not sure whether I don't understand what you're saying, or we're talking about different things. First, what do you mean by "the population mean"? I'm not sure what that could refer to in this context, since the question doesn't pertain to a sample drawn from some larger population. There is no actual "population" involved that could have mean. Second, are you saying that given five measurements, say 149, 153, 147, 152, and 154, SE is the mean of those measurements divided by $\sqrt{5}$, which is 67.5? Isn't SE = $\frac{\sigma}{\sqrt{n}}$, which would be 1.17? $\endgroup$ – Adi Inbar Nov 17 '19 at 0:24

Not the answer you're looking for? Browse other questions tagged or ask your own question.