# Difference between standard error and standard deviation

I'm struggling to understand the difference between the standard error and the standard deviation. How are they different and why do you need to measure the standard error?

• A quick comment, not an answer since two useful ones are already present: standard deviation is a property of the (distribution of the) random variable(s). Standard error is instead related to a measurement on a specific sample. The two can get confused when blurring the distinction between the universe and your sample. – Francesco Jul 15 '12 at 16:57
• Possibly of interest: stats.stackexchange.com/questions/15505/… – Macro Jul 16 '12 at 16:24

To complete the answer to the question, Ocram nicely addressed standard error but did not contrast it to standard deviation and did not mention the dependence on sample size. As a special case for the estimator consider the sample mean. The standard error for the mean is $\sigma \, / \, \sqrt{n}$ where $\sigma$ is the population standard deviation. So in this example we see explicitly how the standard error decreases with increasing sample size. The standard deviation is most often used to refer to the individual observations. So standard deviation describes the variability of the individual observations while standard error shows the variability of the estimator. Good estimators are consistent which means that they converge to the true parameter value. When their standard error decreases to 0 as the sample size increases the estimators are consistent which in most cases happens because the standard error goes to 0 as we see explicitly with the sample mean.

• Re: "...consistent which means their standard error decreases to 0" - that's not true. Do you remember this discussion: stats.stackexchange.com/questions/31036/…? – Macro Jul 15 '12 at 14:27
• Yeah of course I remember the discussion of the unusual exceptions and I was thinking about it when I answered the question. But the question was about standard errors and in simplistic terms the good parameter estimates are consistent and have their standard errors tend to 0 as in the case of the sample mean. – Michael R. Chernick Jul 15 '12 at 14:40
• I agree with your comment - the sample mean's standard error goes to 0 and the sample mean is consistent. But its standard error going to zero isn't a consequence of (or equivalent to) the fact that it is consistent, which is what your answer says. – Macro Jul 15 '12 at 14:53
• @Macro yes the answer could be improved which I decided to do. I think that it is important not to be too technical with the OPs as qualifying everything can be complicated and confusing. But technical accuracy should not be sacrificed for simplicity. So I think the way I addressed this in my edit is the best way to do this. – Michael R. Chernick Jul 15 '12 at 15:02
• I agree it is important not to get technical unless absolutely necessary. My only comment was that, once you've already chosen to introduce the concept of consistency (a technical concept), there's no use in mis-characterizing it in the name of making the answer easier to understand. I think your edit does address my comments though. – Macro Jul 16 '12 at 13:14

Here is a more practical (and not mathematical) answer:

• The SD (standard deviation) quantifies scatter — how much the values vary from one another.
• The SEM (standard error of the mean) quantifies how precisely you know the true mean of the population. It takes into account both the value of the SD and the sample size.
• Both SD and SEM are in the same units -- the units of the data.
• The SEM, by definition, is always smaller than the SD.
• The SEM gets smaller as your samples get larger. This makes sense, because the mean of a large sample is likely to be closer to the true population mean than is the mean of a small sample. With a huge sample, you'll know the value of the mean with a lot of precision even if the data are very scattered.
• The SD does not change predictably as you acquire more data. The SD you compute from a sample is the best possible estimate of the SD of the overall population. As you collect more data, you'll assess the SD of the population with more precision. But you can't predict whether the SD from a larger sample will be bigger or smaller than the SD from a small sample. (This is a simplification, not quite true. See comments below.)

Note that standard errors can be computed for almost any parameter you compute from data, not just the mean. The phrase "the standard error" is a bit ambiguous. The points above refer only to the standard error of the mean.

(From the GraphPad Statistics Guide that I wrote.)

• +1 For clear, useful advice. But some clarifications are in order, of which the most important goes to the last bullet: I would like to challenge you to an SD prediction game. We observe the SD of $n$ iid samples of, say, a Normal distribution. I will predict whether the SD is going to be higher or lower after another $100*n$ samples, say. You pay me a dollar if I'm correct, otherwise I pay you a dollar. (With correct play--which I invite you to figure out!--the expectation of this game is positive for me, getting as high as about $.18$ dollar when $n=2$.) – whuber Jul 16 '12 at 15:11
• @whuber: Of course you are right. It is the variance (SD squared) that won't change predictably as you add more data. The SD will get a bit larger as sample size goes up, especially when you start with tiny samples. This change is tiny compared to the change in the SEM as sample size changes. – Harvey Motulsky Jul 16 '12 at 16:55
• @HarveyMotulsky: Why does the sd increase? – Andrew Nov 28 '12 at 23:43
• With large samples, the sample variance will be quite close to the population variance, so the sample SD will be close to the population SD. With smaller samples, the sample variance will equal the population variance on average, but the discrepancies will be larger. If symmetrical as variances, they will be asymmetrical as SD. Example: Population variance is 100. Two sample variances are 80 or 120 (symmetrical). The sample SD ought to be 10, but will be 8.94 or 10.95. Average sample SDs from a symmetrical distribution around the population variance, and the mean SD will be low, with low N. – Harvey Motulsky Nov 29 '12 at 3:32

Let $\theta$ be your parameter of interest for which you want to make inference. To do this, you have available to you a sample of observations $\mathbf{x} = \{x_1, \ldots, x_n \}$ along with some technique to obtain an estimate of $\theta$, $\hat{\theta}(\mathbf{x})$. In this notation, I have made explicit that $\hat{\theta}(\mathbf{x})$ depends on $\mathbf{x}$. Indeed, if you had had another sample, $\tilde{\mathbf{x}}$, you would have ended up with another estimate, $\hat{\theta}(\tilde{\mathbf{x}})$. This makes $\hat{\theta}(\mathbf{x})$ a realisation of a random variable which I denote $\hat{\theta}$. This random variable is called an estimator. The standard error of $\hat{\theta}(\mathbf{x})$ (=estimate) is the standard deviation of $\hat{\theta}$ (=random variable). It contains the information on how confident you are about your estimate. If it is large, it means that you could have obtained a totally different estimate if you had drawn another sample. The standard error is used to construct confidence intervals.

• Is standard error of estimate equal to standard deviance of estimated variable? – Yurii Jan 3 '16 at 21:59

(note that I'm focusing on standard error of the mean, which I believe the questioner was as well, but you can generate a standard error for any sample statistic)

The standard error is related to the standard deviation but they are not the same thing and increasing sample size does not make them closer together. Rather, it makes them farther apart. The standard deviation of the sample becomes closer to the population standard deviation as sample size increases but not the standard error.

Sometimes the terminology around this is a bit thick to get through.

When you gather a sample and calculate the standard deviation of that sample, as the sample grows in size the estimate of the standard deviation gets more and more accurate. It seems from your question that was what you were thinking about. But also consider that the mean of the sample tends to be closer to the population mean on average. That's critical for understanding the standard error.

The standard error is about what would happen if you got multiple samples of a given size. If you take a sample of 10 you can get some estimate of the mean. Then you take another sample of 10 and new mean estimate, and so on. The standard deviation of the means of those samples is the standard error. Given that you posed your question you can probably see now that if the N is high then the standard error is smaller because the means of samples will be less likely to deviate much from the true value.

To some that sounds kind of miraculous given that you've calculated this from one sample. So, what you could do is bootstrap a standard error through simulation to demonstrate the relationship. In R that would look like:

# the size of a sample
n <- 10
# set true mean and standard deviation values
m <- 50
s <- 100

# now generate lots and lots of samples with mean m and standard deviation s
# and get the means of those samples. Save them in y.
y <- replicate( 10000, mean( rnorm(n, m, s) ) )
# standard deviation of those means
sd(y)
# calcuation of theoretical standard error
s / sqrt(n)


You'll find that those last two commands generate the same number (approximately). You can vary the n, m, and s values and they'll always come out pretty close to each other.

• I found this really helpful, thanks for posting. Would it be fair then to describe the standard error as "the standard deviation of the sampling distribution"? The sampling distribution being y in your code block above? This is what confused me, conflating the sample parameters sd and mean with the sampling distribution parameters. – Doug Fir Jan 2 '18 at 10:26
• If you change your wording to specify sample means for this case, yes. – John Jan 4 '18 at 7:30