Why does the standard deviation not decrease when I do more measurements? [duplicate]

This question already has an answer here:

I made 100 measurements of a certain quantity, calculated mean and standard deviation (with MySQL), and got mean=0.58, SD=0.34.

The std seemed too high relative to the mean, so I made 1000 measurements. This time I got mean=0.572, SD=0.33.

I got frustrated by the high standard deviation, so I made 10,000 measurements. I got mean=0.5711, SD=0.34.

I thought maybe this was a bug in MySQL, so I tried to use the Excel functions, but got the same results.

Why does the standard deviation remain high even though I do so many measurements?

marked as duplicate by Nick Cox, Glen_b, whuber♦Mar 11 '14 at 12:00

The standard deviation is a measurement of the "spread" of your data. The analogy I like to use is target shooting. If you're an accurate shooter, your shots cluster very tightly around the bullseye (small standard deviation). If you're not accurate, they are more spread out (large standard deviation).

Some data is fundamentally "all over the place", and some is fundamentally tightly clustered about the mean.

If you take more measurements, you are getting a more accurate picture of the spread. You shouldn't expect to get less spread--just less error in your measurement of a fundamental characteristic of the data.

If you have an inaccurate shooter take five shots, and an accurate shooter take five shots, you will get a not-too-reliable idea of their accuracy. Maybe the inaccurate shooter got lucky a few times, so the pattern is tighter than you would expect from him over the long haul. Similarly, maybe you caught the accurate shooter at a bad time and just happened to get two bad shots in the five, skewing the results.

If, instead, you have them each take a thousand shots, then you will be much more confident that you are getting a good look at their actual accuracy. It's not the accuracy of the shooter changing as you get more data--it's the confidence you have in the picture you are getting of their accuracy.

This question seems trivial to statisticians, but I managed to make this mistake twice, and after a colleague of mine also made the same mistake, I decided to write the answer, in order to help myself and other people making the same mistake.

The standard deviation does not become lower when the number of measurements grows.. The standard deviation is just the square root of the average of the square distance of measurements from the mean.

So, for example, if the "real" value that is measured is 1, half of the measurements are 1.05, and half of the measurements are 0.95, then the mean will be 1, which is just the correct value, and the std will be 0.05, regardless of the number of experiments.

The thing that does become lower when the number of measurements grows is the confidence interval, which is inversely proportional to the square root of the number of measurements. For example, the radius of the 95% confidence interval is approximately:

$$1.96 \cdot \frac{SD({\rm Measurements})}{\sqrt{{\rm Count(Measurements)}}}$$

So, the question comes from confusing between the standard deviation and the confidence interval.

• 1. You might want to change "STD" to "SD" which is more standard and STD had another meaning (to medical people, STD means sexually transmitted disease). 2. Your final equation isn't quite right. The multiplier is not 1.96 but rather is a constant from the t distribution. With large n, it is 1.96 but with smaller n that multiplier is larger. – Harvey Motulsky Mar 10 '14 at 14:18
• You might like to look into standard error as well. The standard deviation of the sample doesn't decrease, but the standard error, which is the standard deviation of the sampling distribution of the mean, does decrease. The standard error is the fraction in your answer that you multiply by 1.96. (You can search for the standard error on this site using the tag standard-error.) – TooTone Mar 10 '14 at 14:20
• You might need to clarify that its the sample standard deviation (i.e., with bias adjustment) that is an unbiased estimator of the population standard deviation. – John Mar 10 '14 at 16:55
• @John It isn't. The (n-1)-denominator version of variance is unbiased for population variance. The corresponding standard deviation is biased. – Glen_b Mar 10 '14 at 17:29
• Looks like you're right @Glen_b. My mistake. – John Mar 10 '14 at 17:32

The mean and standard deviation are population properties. As you increase your number of observations you will on average get more precise estimates from your sample for both the population mean and standard deviation.

It sounds like you are confusing the standard error of the mean with the standard deviation. The standard error of the mean is the standard deviation of your estimate of the mean. The standard error of the mean (i.e., the precision of your estimate of the mean) does get smaller as sample size increases.

http://en.wikipedia.org/wiki/Standard_error#Standard_error_of_the_mean