# Standard Error, Standard Deviation and Variance confusion

I am quite confused in these terminologies (especially but not limited to regression)
I do understand what Variance and Standard Deviation means, they measure the dispersion / variability of the data.

However, according to my understanding, Standard Error $$= \frac{s}{\sqrt{n}}$$ where s is the sample standard deviation.
But in regression (for simplicity, here refer to Simple Linear Regression but MLR shall be of the same fashion) $$y = \beta_0 + \beta_{1}x + e$$.
Variance of $$\hat\beta_1$$ = $$\frac{\sigma^2}{SXX}$$
And while we are doing confidence interval for $$\hat\beta_1$$, the SE we use is simply the square root of Var($$\hat\beta_1$$) = $$\frac{\sigma}{\sqrt{SXX}}$$ without needing to divide by $$\sqrt{n}$$

My questions:
1) Is there a difference between normal Standard Error (of mean) that we talk about (i.e. $$\frac{s}{\sqrt{n}}$$) and the SE we talk in regression.
2) I suppose, $$\hat\beta_1$$ is not a mean but purely an estimator of the parameter $$\beta_1$$, so why do we use SE when we are constructing confidence interval of $$\hat\beta_1$$?
3) What about confidence interval for predicted $$y$$ value and fitted $$y$$ value respectively?

• Whenever you speak of the standard deviation of an estimator that is a standard error. For example $\bar X$ is an estimator of $\mu$ so $SD(\bar X) = SE(\bar X) = \sigma/\sqrt{n}.$ Often, if $\sigma$ is unknown and estimated by $S$ authors say that the 'standard error' of $\bar X$ is $S/\sqrt{n}$ when more fastidious terminology would be 'estimated standard error'. (The word estimated gets dropped, usually with no harm, because one knows $S$ is an estimate.) Nov 5, 2020 at 16:30

The term "standard error" refers to the standard deviation of a statistic that is calculated. So, you can calculate a standard error for a mean--because the mean is a statistic. You can also calculate a standard error for a parameter estimate like $$\hat{\beta}$$.

We say standard error instead of standard deviation to distinguish between a value that's calculated from repeated observations and an estimate that's based on a theory about the distribution.

We only have one observation for $$\hat{\beta}$$, and we have mathematical theory to derive its sampling error--so we call that the standard error.

We have more than one observation of a variable X, and we calculate the sampling error based on that observed data--so we call that statistic the standard deviation.

• so the formula $\frac{s}{\sqrt{n}}$ is just the formula for standard error of the mean ($\bar{x}$)
– dust
Nov 5, 2020 at 16:41
• and the difference between standard deviation and standard error is the number of observations. If the data is a statistic, which has only one observation, summarizing a bunch of data point (like the mean summarize the whole set of data) then it will be called standard error. While if it is a lot of observations (like the set of data itself), then it will be the standard deviation?
– dust
Nov 5, 2020 at 16:44
• Yes--That's the gist of the reasoning behind having two different terms for what looks like the same thing. Nov 18, 2020 at 18:36

The terminology is the same everywhere in statistics I think:

• Variance $$\sigma^2$$ is the second moment of a known probability distribution
• Standard Deviation $$\sigma$$ is the square root of variance
• Variance of the mean $$\sigma^2_{\mu} = \frac{\sigma^2}{N}$$ is the variance of the mean of $$N$$ i.i.d random variables
• Standard Deviation of the Mean $$\sigma_{\mu}$$ is the square root of the variance of the mean

The 4 above metrics apply analytically to probability distributions. One can estimate any one of them, typically denoted by letter $$s$$ and prefix 'sample', such as 'sample error of the mean' $$s_{\mu}$$. Sample standard deviation and Sample standard deviation of the mean are also known as Standard Error and Standard Error of the mean (SEM) respectively

• Variance and standard deviation are metrics of the distribution of the random variables in analytic case and a metric of data in the sample case. These terms are not applicable to parameters of your model, such as $$\beta$$ or $$\hat \beta$$. These are simply the parameter and its estimate.
• Confidence intervals only apply to unknown parameters of the model, they do not apply to parts of data such as $$y$$. The closest entity to a confidence interval when applied to random variable itself is a tolerance interval, namely, the interval where the random variable is likely to fall given the exact model parameters