5
$\begingroup$

The standard error of an estimator is defined as the square root of the the estimator variance (or mean squared error, MSE, for unbiased estimators). More specifically, if we wanted to get the standard error of the sample mean $\bar{X}$, we would divide the variance of the sample used to calculate $\bar{X}$ by the square root of $n$:
$$ {\rm MSE}(\bar{X}) = \frac{s^2}{n}, \\ {\rm and,} \\ ~ \\ {\rm S.E.} (\bar{X}) = \sqrt\frac{s^2}{n} = \frac{s}{\sqrt{n}} $$ where $s^2$ is the sample variance for the sample of $X$s.

However, in OLS regressions, the standard error of the regression is defined as: $$ \sqrt\frac{SSR}{n - K} $$ where SSR is the sum of squared residuals.

My question is: does this not need to be corrected again by dividing by the sample size $n$?

Surely $\sqrt{^{SSR}/_{n - K}}$ is just the sample estimate of the population variance and, according to the formula above, it needs to be corrected. I understand that in both cases the standard error is the square root of the MSE but in the first case the MSE is the sample variance that the estimator came from divided by $n$ and in the second example the MSE is just the sample variance. Does anyone have an explanation?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Standard error of the regression is the estimate of the standar deviation of the regression disturbance. Suppose we have the regression model

$$Y_t=X_t'\beta+u_t,$$

where $Eu_t^2=\sigma^2$. Then the standard error of the regression is the estimate of $\sigma$, i.e. $\hat\sigma$. This is an estimate similar to $\bar X$ and you can talk about its MSE. But you do not need to divide it by $n$ as you do not divide $\bar X$ by $n$.

It is possible to show that MSE of $\hat\sigma$ is a a quantity bounded in probability divided by $\sqrt{n}$ and goes to zero similar to MSE of $\bar X$.

Improve this answer
$\endgroup$
13
  • $\begingroup$ ,Thanks for the quick reply. In the above case I wasn't meaning to divide $\bar{x}$ by $n$, just the sample variance of the set that it is derived from (just the general standard error of the mean formula that you see in text books). I just think that if we were to apply this formula directly to the OLS case we would get the sample variance of the errors, divide them by the sample size and then call this the variance of the OLS regression. But in the text books, the variance of the OLS regression is just the sample variance of the errors. I can't seem to reconcile them in my head ... $\endgroup$
    – EconStats
    Oct 7, 2013 at 18:39
  • 1
    $\begingroup$ I believe that instead of writing "standar error of the regression disturbance" [sic] you may have intended "standard deviation of the ... disturbance." This little slip, if not corrected, will just add to the confusion! $\endgroup$
    – whuber ♦
    Oct 7, 2013 at 20:31
  • $\begingroup$ @whuber Thanks for pitching in! The common factor between my two examples is that the standard error of the estimates is always the square root of the variances of the estimates so I understand the intuition behind them . . I just wish the math was more obvious! $\endgroup$
    – EconStats
    Oct 7, 2013 at 22:58
  • $\begingroup$ Standard error of regression is a term it is not a standard error of a regression. Hence the confusion. Think about it, sample mean is a number, which can be thought as a random variable. So we can talk about its standard error. What is a regression? It is a concept, a term describing a way to get certain estimates it is not an estimate. So your intuition is wrong in this case. $\endgroup$
    – mpiktas
    Oct 8, 2013 at 2:49
  • 1
    $\begingroup$ Dividing by $n$ happens (often) when you take the variance of an estimator. The object you are considering, $\sqrt{\textrm{SSR}/(n-k)}$, is not the variance of an estimator. Rather, it is an estimator of the standard deviation of the regression error. It is analogous to $\overline{X}$ not to $V(\overline{X})$. Closely related is the estimator of the variance of the regression error, $\hat{\sigma}^2_{OLS} = \textrm{SSR}/(n-k)$. When the regression error is distributed normal, the variance of this estimator is $V(\hat{\sigma}^2_{OLS})=2\sigma^4/(n-k)$. There, the $n$ in the denom is back! $\endgroup$
    – Bill
    Oct 8, 2013 at 14:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.