3
$\begingroup$

The standard error of an estimator is defined as the square root of the the estimator variance (or mean squared error, MSE, for unbiased estimators). More specifically, if we wanted to get the standard error of the sample mean $\bar{X}$, we would divide the variance of the sample used to calculate $\bar{X}$ by the square root of $n$:
$$ {\rm MSE}(\bar{X}) = \frac{s^2}{n}, \\ {\rm and,} \\ ~ \\ {\rm S.E.} (\bar{X}) = \sqrt\frac{s^2}{n} = \frac{s}{\sqrt{n}} $$ where $s^2$ is the sample variance for the sample of $X$s.

However, in OLS regressions, the standard error of the regression is defined as: $$ \sqrt\frac{SSR}{n - K} $$ where SSR is the sum of squared residuals.

My question is: does this not need to be corrected again by dividing by the sample size $n$?

Surely $\sqrt{^{SSR}/_{n - K}}$ is just the sample estimate of the population variance and, according to the formula above, it needs to be corrected. I understand that in both cases the standard error is the square root of the MSE but in the first case the MSE is the sample variance that the estimator came from divided by $n$ and in the second example the MSE is just the sample variance. Does anyone have an explanation?

$\endgroup$
2
$\begingroup$

Standard error of the regression is the estimate of the standar deviation of the regression disturbance. Suppose we have the regression model

$$Y_t=X_t'\beta+u_t,$$

where $Eu_t^2=\sigma^2$. Then the standard error of the regression is the estimate of $\sigma$, i.e. $\hat\sigma$. This is an estimate similar to $\bar X$ and you can talk about its MSE. But you do not need to divide it by $n$ as you do not divide $\bar X$ by $n$.

It is possible to show that MSE of $\hat\sigma$ is a a quantity bounded in probability divided by $\sqrt{n}$ and goes to zero similar to MSE of $\bar X$.

$\endgroup$
  • $\begingroup$ ,Thanks for the quick reply. In the above case I wasn't meaning to divide $\bar{x}$ by $n$, just the sample variance of the set that it is derived from (just the general standard error of the mean formula that you see in text books). I just think that if we were to apply this formula directly to the OLS case we would get the sample variance of the errors, divide them by the sample size and then call this the variance of the OLS regression. But in the text books, the variance of the OLS regression is just the sample variance of the errors. I can't seem to reconcile them in my head ... $\endgroup$ – EconStats Oct 7 '13 at 18:39
  • 1
    $\begingroup$ I believe that instead of writing "standar error of the regression disturbance" [sic] you may have intended "standard deviation of the ... disturbance." This little slip, if not corrected, will just add to the confusion! $\endgroup$ – whuber Oct 7 '13 at 20:31
  • $\begingroup$ @whuber Thanks for pitching in! The common factor between my two examples is that the standard error of the estimates is always the square root of the variances of the estimates so I understand the intuition behind them . . I just wish the math was more obvious! $\endgroup$ – EconStats Oct 7 '13 at 22:58
  • $\begingroup$ Standard error of regression is a term it is not a standard error of a regression. Hence the confusion. Think about it, sample mean is a number, which can be thought as a random variable. So we can talk about its standard error. What is a regression? It is a concept, a term describing a way to get certain estimates it is not an estimate. So your intuition is wrong in this case. $\endgroup$ – mpiktas Oct 8 '13 at 2:49
  • 1
    $\begingroup$ Dividing by $n$ happens (often) when you take the variance of an estimator. The object you are considering, $\sqrt{\textrm{SSR}/(n-k)}$, is not the variance of an estimator. Rather, it is an estimator of the standard deviation of the regression error. It is analogous to $\overline{X}$ not to $V(\overline{X})$. Closely related is the estimator of the variance of the regression error, $\hat{\sigma}^2_{OLS} = \textrm{SSR}/(n-k)$. When the regression error is distributed normal, the variance of this estimator is $V(\hat{\sigma}^2_{OLS})=2\sigma^4/(n-k)$. There, the $n$ in the denom is back! $\endgroup$ – Bill Oct 8 '13 at 14:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.