What is the difference between population standard deviation, sample standard deviation, and standard error?

Question

Sorry for the rudimentary question, but I just want to make sure I understand everything well conceptually. I understand how we get the standard deviation of a population. My questions are as follows:

1. If we want to describe the spread of a sample of data, why would we not use the same formula we would for the population? In other words, given a population of 20 individuals and a sample of 20 individuals, why don't we divide by N for both of the data sets to express how far on average each data point is from each sample's mean? Given all data points are the same, wouldn't these two necessarily have the same spread, and therefore should have the same numerical value for a measure of spread (standard deviation)?

2. Can one use the sample standard deviation to estimate the population standard deviation? Is this when the N vs. n-1 question comes into play?

3. Standard error tells us how far, on average, a given sample mean deviates from the true mean of these means (which will be the population mean), correct?

Answer 1

1. You rarely have entire population data at hand. That is why random samples are used to approximate population characteristics. Using population-based formulae with your sample without a correction will yield biased estimates.

2. N-1 is simply a correction applied to take into account the fact that you are working with a sample, rather than the population itself

3. Yes (that's called an empirical standard error)

Important note: I think you will gain further insight into the terms and properties of standard error, standard deviation, and population corrections by reading the following threads

1. Difference between standard error and standard deviation

2. Intuition behind applying N-1 correction for sample variance

Answer 2

As a student, this question also stumped me for a while, but I believe I have some clarity now.

• Consider sampling $n$ values $\left\{Y_1, ..., Y_n\right\}$, with replacement, from a population $Y$, whose whole distribution we might not know, but we do know its expected value is $E[Y] = \mu$ and its variance is $Var[Y] = \sigma^2$. If you assume $Y$ is a discrete set of $N$ values $\left\{V_1, ..., V_N\right\}$, we can write $$Var[Y] = E[(Y - \mu)^2] = \sum_{i = 1}^N \frac{1}{N} \cdot (V_i - \mu)^2 =\sigma^2.$$

• Nothing's wrong with calculating statistics from your sample data - at least we're getting some insight into the data, even though it might be biased or whatever. We can get the sample's basic statistics as follows: $$\textbf{Sample's mean}: \overline{Y} = \frac{1}{n} \sum_{i = 1}^n Y_i.$$ $$\textbf{Sample's variance}: \overline{\sigma_Y}^2 = \frac{1}{n} \sum_{i = 1}^n (Y_i - \overline{Y})^2.$$ However, the implicit assumption is that you're trying to use statistics from the sample data to estimate statistics of the population. Using the sample's variance turns out to be a little biased when applied to estimating the population variance. The derivation can be found at https://en.wikipedia.org/wiki/Variance#Biased_sample_variance, keeping in mind the simple fact that any sample $Y_i$ must satisfy $E[Y_i] = \mu$, $Var[Y_i] = \sigma^2$. (Of course, once we observe what $Y_i$ is, it's no longer a random variable.) We have: $$E[\overline{\sigma_Y}^2] = \frac{n-1}{n} \sigma^2.$$

• However, this is simple to correct. The sample variance (not sample's variance) is an unbiased estimator of the population variance, and is given by $$s^2 = \frac{n}{n-1} \overline{\sigma_Y}^2 = \frac{1}{n-1} \sum_{i = 1}^n (Y_i - \overline{Y})^2,$$ where $$E[s^2] = \sigma^2.$$

• So why do we need $n-1$? It comes from the fact that when we are sampling, we assume sampling with replacement. Suppose the population $Y$ was a set of 20 distinct elements. A random sample of 20 elements with replacement might not get you the population set; you could have repeats. In fact, if you're really unlucky, you could get 20 copies of the same element, with sample's variance 0. You could also get lucky and get a sample's variance higher than the population variance, if you only sample the endpoints. All that Bessel's correction tells you is that the expected value of the sample's variance skews a bit low compared to the population variance, and scales with the sample size $n$.

Hope this helps!

Answer 3

Population Standard Deviation - Fixed measure of spread of an underlying distribution that you are trying to probe statistically

Sample Standard Deviation - A corresponding measure of spread calculated from samples of the underlying distribution. Used as an estimator for the "true" population SD.

Standard Error - The measure of spread of the estimator itself. In the case of the standard error of the estimator of the mean, we see that the standard error goes down as the number of samples grows. Therefore, we see that (for an unbiased, consistent estimator) that we converge to the true underlying distribution value of the mean.