So I am a bit confused about the interpretation of the standard error especially when the population standard deviation is unknown. What I've constructed from what I've read about it is that: the mean of the sampling distribution is the population mean and the SD of the sampling distribution is the population standard deviation (sigma) divided by the square root of the sample size. By taking many samples of a reasonable size from a population with a known SD, we can calculate a 95% CI for the mean of the sampling distribution (or is it the mean of the population?). In this case, each sample has the same SE ( which is equal to the SD of the sampling distribution) because we use the same population SD. Is this right?

If the population SD is unknown, we use the sample SD (s) and now each sample has a different SE. By taking many samples of a reasonable size and taking the mean of the means of the sample, we can get an estimate for the mean of the sampling distribution. To calculate the CI, we need the SE which is the sample standard deviation of the sample means divided by square root of the sample size. Is my understanding correct or am I missing something? What exactly do we estimate with the the samples when we don't know sigma and how is the SE interpreted differently from when sigma is known.

  • $\begingroup$ You have the terminology almost right, but maybe with a bit of confusion. I hope my answer helps to get rid of the confusion. $\endgroup$ – BruceET Jun 17 '20 at 4:50

Suppose you want to estimate the mean $\mu$ of a normal population using the mean $\bar X$ of a random sample $X_1, X_2, \dots, X_n$ of size $n$ from the population.

The term 'standard error' usually refers to the the standard deviation of an estimator. In the current situation the standard error of $\bar X$ is $\sigma/\sqrt{n},$ where $\sigma$ is the standard deviation of the population.

However, if $\sigma$ is unknown and estimated by the sample standard deviation $S= \sqrt{\frac{1}{n-1}\sum_{i=1}^n (X_i - \bar X)^2},$ then $T = \frac{\bar X = \mu}{S/\sqrt{n}}$ has Student's t distribution with $\nu = n-1$ degrees of freedom, $\mathsf{T}(\nu).$

This fact is used to make a 95% confidence for (estimating) $\mu$ of the form $\bar X \pm t^*\frac{S}{\sqrt{n}},$ where $t^*$ cuts probability $0.025= 2.5\%$ from the upper tail of the distribution $\mathsf{T}(\nu).$ [By the symmetry of the t distribution $-t^*$ cuts 2.5% of the probability from the lower tail of the distribution, leaving 95% in the middle.]

In this situation the estimated standard deviation of $\bar X$ is $S/\sqrt{n}.$ Usually the word estimated is left out because as soon as you see $S$ you know it's an estimate of $\sigma.$

You might say you have two standard errors of the sample mean $\bar X,$ the theoretical one $\sigma/\sqrt{n},$ and the estimated one, $S/\sqrt{n}.$ However, if $\sigma$ is unknown, only the (estimated) standard error $S/\sqrt{n}$ is of practical use---in making a confidence interval as above or in testing a hypothesis about $\mu.$

Note: There are situations in which $\mu$ is unknown and $\sigma$ is known. Then a 95% confidence interval for $\mu$ is $\bar X \pm 1.96 \frac{\sigma}{\sqrt{n}},$ where 1.96 cuts probability 2.5% from the upper tail of the standard normal distribution. (This confidence interval is based on the fact that $Z = \frac{\bar X - \mu}{\sigma/\sqrt{n}}$ has a standard normal distribution.) Notice that the theoretical standard error $\sigma/\sqrt{n}$ is used when $\sigma$ is known.

  • $\begingroup$ i see. but is it right to use a single sample to estimate the mu? is the estimation any more reliable for when sigma is known as compared to when it is unknown? say we take a sample from a population and calculate its mean and a 95% CI (xbar = 5, CI = 3 - 7) . we can say that the population mean is 5 give or take 2 and this interval will capture the true population mean 95% of the time. correct? $\endgroup$ – NoLifeKing Jun 17 '20 at 15:10
  • 1
    $\begingroup$ Depending on the size of the sample, a single sample may be good enough. Estimation is more reliable if $\sigma$ is known. Over the long run 95% confidence intervals will capture (include) the true value of $\mu$ 95% of the time, but nothing can be said for sure about the confidence interval from any one sample.. $\endgroup$ – BruceET Jun 17 '20 at 16:19
  • 1
    $\begingroup$ Thanks, it makes a lot more sense now. $\endgroup$ – NoLifeKing Jun 20 '20 at 15:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.