Standard Error, Standard Deviation and Variance confusion I am quite confused in these terminologies (especially but not limited to regression)
 I do understand what Variance and Standard Deviation means, they measure the dispersion / variability of the data.
 However, according to my understanding, Standard Error $= \frac{s}{\sqrt{n}}$ where s is the sample standard deviation.
 But in regression (for simplicity, here refer to Simple Linear Regression but MLR shall be of the same fashion) $y = \beta_0 + \beta_{1}x + e$.
 Variance of $\hat\beta_1$ = $\frac{\sigma^2}{SXX}$
 And while we are doing confidence interval for $\hat\beta_1$, the SE we use is simply the square root of Var($\hat\beta_1$) = $\frac{\sigma}{\sqrt{SXX}}$ without needing to divide by $\sqrt{n}$
My questions:
 1) Is there a difference between normal Standard Error (of mean) that we talk about (i.e. $\frac{s}{\sqrt{n}}$) and the SE we talk in regression.
 2) I suppose, $\hat\beta_1$ is not a mean but purely an estimator of the parameter $\beta_1$, so why do we use SE when we are constructing confidence interval of $\hat\beta_1$?
 3) What about confidence interval for predicted $y$ value and fitted $y$ value respectively?
 A: The term "standard error" refers to the standard deviation of a statistic that is calculated. So, you can calculate a standard error for a mean--because the mean is a statistic. You can also calculate a standard error for a parameter estimate like $\hat{\beta}$.
We say standard error instead of standard deviation to distinguish between a value that's calculated from repeated observations and an estimate that's based on a theory about the distribution.
We only have one observation for $\hat{\beta}$, and we have mathematical theory to derive its sampling error--so we call that the standard error.
We have more than one observation of a variable X, and we calculate the sampling error based on that observed data--so we call that statistic the standard deviation.
A: The terminology is the same everywhere in statistics I think:

*

*Variance $\sigma^2$ is the second moment of a known probability distribution

*Standard Deviation $\sigma$ is the square root of variance

*Variance of the mean $\sigma^2_{\mu} = \frac{\sigma^2}{N}$ is the variance of the mean of $N$ i.i.d random variables

*Standard Deviation of the Mean $\sigma_{\mu}$ is the square root of the variance of the mean

The 4 above metrics apply analytically to probability distributions. One can estimate any one of them, typically denoted by letter $s$ and prefix 'sample', such as 'sample error of the mean' $s_{\mu}$. Sample standard deviation and Sample standard deviation of the mean are also known as Standard Error and Standard Error of the mean (SEM) respectively
With respect to your questions:

*

*Variance and standard deviation are metrics of the distribution of the random variables in analytic case and a metric of data in the sample case. These terms are not applicable to parameters of your model, such as $\beta$ or $\hat \beta$. These are simply the parameter and its estimate.

*When you construct a confidence interval for an unknown
parameter, you perform a hypothesis test. The confidence interval is likely to be a function of the moments of the distribution, or their sample counterparts, but that depends strongly on the underlying distribution.

*Confidence intervals only apply to unknown parameters of the model, they do not apply to parts of data such as $y$. The closest entity to a confidence interval when applied to random variable itself is a tolerance interval, namely, the interval where the random variable is likely to fall given the exact model parameters

