14
$\begingroup$

I made 100 measurements of a certain quantity, calculated mean and standard deviation (with MySQL), and got mean=0.58, SD=0.34.

The std seemed too high relative to the mean, so I made 1000 measurements. This time I got mean=0.572, SD=0.33.

I got frustrated by the high standard deviation, so I made 10,000 measurements. I got mean=0.5711, SD=0.34.

I thought maybe this was a bug in MySQL, so I tried to use the Excel functions, but got the same results.

Why does the standard deviation remain high even though I do so many measurements?

Improve this question
$\endgroup$
0
13
$\begingroup$

The standard deviation is a measurement of the "spread" of your data. The analogy I like to use is target shooting. If you're an accurate shooter, your shots cluster very tightly around the bullseye (small standard deviation). If you're not accurate, they are more spread out (large standard deviation).

Some data is fundamentally "all over the place", and some is fundamentally tightly clustered about the mean.

If you take more measurements, you are getting a more accurate picture of the spread. You shouldn't expect to get less spread--just less error in your measurement of a fundamental characteristic of the data.

If you have an inaccurate shooter take five shots, and an accurate shooter take five shots, you will get a not-too-reliable idea of their accuracy. Maybe the inaccurate shooter got lucky a few times, so the pattern is tighter than you would expect from him over the long haul. Similarly, maybe you caught the accurate shooter at a bad time and just happened to get two bad shots in the five, skewing the results.

If, instead, you have them each take a thousand shots, then you will be much more confident that you are getting a good look at their actual accuracy. It's not the accuracy of the shooter changing as you get more data--it's the confidence you have in the picture you are getting of their accuracy.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Very nice answer. I have a little confusion. Is it true that random error decreases as more measurements are made? But if this is true then standard deviation should also decrease. Thanks in advance. $\endgroup$ – ado sar Jan 17 '20 at 13:06
  • $\begingroup$ I think the thing you are talking about going down may be standard error of the mean? see stats.stackexchange.com/a/32385/41662 (more subtle thing where you're more likely to have a skew of outliers when you have a few measurments (you haven't taken enough measurements that the effect of "most of the values will be near the mean" has been enough yet). So it might be that you converge on the actual standard deviation of the data by starting high and getting lower as you get closer to it. But the random thing's "actual" standard deviation doesn't change, just your picture of it.)? $\endgroup$ – msouth Jan 31 '20 at 20:31
  • $\begingroup$ On second thought I'm even less sure about the long parenthetical--I'm not familiar enough with the theory to know what to expect of the measured SD of the data as number of measurements increases--maybe your few measurements are more likely to see things near the mean and thus show you a smaller value of measured SD, until you do take enough samples to get the outliers in there reliably. See comments on the answer I linked, I think it might be addressed there. But I think your question is confusing the expected error in your measurement of the SD with the SD of the data itself. $\endgroup$ – msouth Jan 31 '20 at 20:45
15
$\begingroup$

This question seems trivial to statisticians, but I managed to make this mistake twice, and after a colleague of mine also made the same mistake, I decided to write the answer, in order to help myself and other people making the same mistake.

The standard deviation does not become lower when the number of measurements grows.. The standard deviation is just the square root of the average of the square distance of measurements from the mean.

So, for example, if the "real" value that is measured is 1, half of the measurements are 1.05, and half of the measurements are 0.95, then the mean will be 1, which is just the correct value, and the std will be 0.05, regardless of the number of experiments.

The thing that does become lower when the number of measurements grows is the confidence interval, which is inversely proportional to the square root of the number of measurements. For example, the radius of the 95% confidence interval is approximately:

$$1.96 \cdot \frac{SD({\rm Measurements})}{\sqrt{{\rm Count(Measurements)}}}$$

So, the question comes from confusing between the standard deviation and the confidence interval.

Improve this answer
$\endgroup$
6
  • 5
    $\begingroup$ 1. You might want to change "STD" to "SD" which is more standard and STD had another meaning (to medical people, STD means sexually transmitted disease). 2. Your final equation isn't quite right. The multiplier is not 1.96 but rather is a constant from the t distribution. With large n, it is 1.96 but with smaller n that multiplier is larger. $\endgroup$ – Harvey Motulsky Mar 10 '14 at 14:18
  • 6
    $\begingroup$ You might like to look into standard error as well. The standard deviation of the sample doesn't decrease, but the standard error, which is the standard deviation of the sampling distribution of the mean, does decrease. The standard error is the fraction in your answer that you multiply by 1.96. (You can search for the standard error on this site using the tag standard-error.) $\endgroup$ – TooTone Mar 10 '14 at 14:20
  • $\begingroup$ You might need to clarify that its the sample standard deviation (i.e., with bias adjustment) that is an unbiased estimator of the population standard deviation. $\endgroup$ – John Mar 10 '14 at 16:55
  • 1
    $\begingroup$ @John It isn't. The (n-1)-denominator version of variance is unbiased for population variance. The corresponding standard deviation is biased. $\endgroup$ – Glen_b Mar 10 '14 at 17:29
  • $\begingroup$ Looks like you're right @Glen_b. My mistake. $\endgroup$ – John Mar 10 '14 at 17:32
10
$\begingroup$

The mean and standard deviation are population properties. As you increase your number of observations you will on average get more precise estimates from your sample for both the population mean and standard deviation.

It sounds like you are confusing the standard error of the mean with the standard deviation. The standard error of the mean is the standard deviation of your estimate of the mean. The standard error of the mean (i.e., the precision of your estimate of the mean) does get smaller as sample size increases.

http://en.wikipedia.org/wiki/Standard_error#Standard_error_of_the_mean

Improve this answer
$\endgroup$
1
  • $\begingroup$ In the question I believe "mean" and "SD" are actually "sample mean" and "sample SD" $\endgroup$ – ten1o Feb 10 at 11:23

Not the answer you're looking for? Browse other questions tagged or ask your own question.