clarification about the standard error when sigma is unknown

Question

So I am a bit confused about the interpretation of the standard error especially when the population standard deviation is unknown. What I've constructed from what I've read about it is that: the mean of the sampling distribution is the population mean and the SD of the sampling distribution is the population standard deviation (sigma) divided by the square root of the sample size. By taking many samples of a reasonable size from a population with a known SD, we can calculate a 95% CI for the mean of the sampling distribution (or is it the mean of the population?). In this case, each sample has the same SE ( which is equal to the SD of the sampling distribution) because we use the same population SD. Is this right?

If the population SD is unknown, we use the sample SD (s) and now each sample has a different SE. By taking many samples of a reasonable size and taking the mean of the means of the sample, we can get an estimate for the mean of the sampling distribution. To calculate the CI, we need the SE which is the sample standard deviation of the sample means divided by square root of the sample size. Is my understanding correct or am I missing something? What exactly do we estimate with the the samples when we don't know sigma and how is the SE interpreted differently from when sigma is known.

Answer 1

Suppose you want to estimate the mean $\mu$ of a normal population using the mean $\bar X$ of a random sample $X_1, X_2, \dots, X_n$ of size $n$ from the population.

The term 'standard error' usually refers to the the standard deviation of an estimator. In the current situation the standard error of $\bar X$ is $\sigma/\sqrt{n},$ where $\sigma$ is the standard deviation of the population.

However, if $\sigma$ is unknown and estimated by the sample standard deviation $S= \sqrt{\frac{1}{n-1}\sum_{i=1}^n (X_i - \bar X)^2},$ then $T = \frac{\bar X = \mu}{S/\sqrt{n}}$ has Student's t distribution with $\nu = n-1$ degrees of freedom, $\mathsf{T}(\nu).$

This fact is used to make a 95% confidence for (estimating) $\mu$ of the form $\bar X \pm t^*\frac{S}{\sqrt{n}},$ where $t^*$ cuts probability $0.025= 2.5\%$ from the upper tail of the distribution $\mathsf{T}(\nu).$ [By the symmetry of the t distribution $-t^*$ cuts 2.5% of the probability from the lower tail of the distribution, leaving 95% in the middle.]

In this situation the estimated standard deviation of $\bar X$ is $S/\sqrt{n}.$ Usually the word estimated is left out because as soon as you see $S$ you know it's an estimate of $\sigma.$

You might say you have two standard errors of the sample mean $\bar X,$ the theoretical one $\sigma/\sqrt{n},$ and the estimated one, $S/\sqrt{n}.$ However, if $\sigma$ is unknown, only the (estimated) standard error $S/\sqrt{n}$ is of practical use---in making a confidence interval as above or in testing a hypothesis about $\mu.$

---

Note: There are situations in which $\mu$ is unknown and $\sigma$ is known. Then a 95% confidence interval for $\mu$ is $\bar X \pm 1.96 \frac{\sigma}{\sqrt{n}},$ where 1.96 cuts probability 2.5% from the upper tail of the standard normal distribution. (This confidence interval is based on the fact that $Z = \frac{\bar X - \mu}{\sigma/\sqrt{n}}$ has a standard normal distribution.) Notice that the theoretical standard error $\sigma/\sqrt{n}$ is used when $\sigma$ is known.