Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
The standard error of the mean is often calculated with the formula $\sqrt{\sigma^2 / n}$, which uses population variance (or an estimate of it).
However, standard error of the mean is defined as the standard deviation of the sample means. So why don't we ever directly calculate standard error of the mean as $\sqrt{E[(X_i - \overline X_n)^2]}$ where $X_i$ is a sample mean and $\mu$ is the mean of the sample means? Since the formula for standard deviation is $\sqrt{E[(X - \mu)^2]}$.
Well, typically, we only have a single realization of the sample mean to work with - we are not often in a situation where we have several datasets drawn from the same population. Even if we did have multiple samples, why not just combine them to get an even more precise estimate of the sample mean, given that we likely care more about estimating the quantity itself?
However, resampling techniques use this approach to estimate standard errors. The most familiar is likely the bootstrap procedure, which can be used to estimate the standard error of any statistic. If you are unfamiliar, bootstrapping returns many resampled sample statistics, which we take the standard deviation of to estimate the standard error of that statistic.
The "standard error" of an estimator is just the standard deviation of that estimator. Typically this depends on the unknown parameters in the model, and so the true standard error is a function of the parameters and is unknown. We can then estimate the true standard error to get the "estimated standard error" which is now a statistic that depends only on the observed data. In the context of using a sample mean as an estimator for the population mean, we have:
$$\begin{align} \text{True standard error} &= \mathbb{S}(\bar{X}_n) = \frac{\sigma}{\sqrt{n}}, \\[12pt] \text{Estimated standard error} &= \hat{\mathbb{S}}(\bar{X}_n) = \frac{s_n}{\sqrt{n}}. \\[6pt] \end{align}$$
As you can see, the estimated standard error in this problem does indeed use the sample standard deviation instead of the true standard deviation in the model (though it is still different to what you're proposing, which does not seem to apply the correct scaling at all).
