# Standard error in linear regression

Lets say I run a simple linear regression. The coefficient estimate is 2, the standard error of the coefficient is 0.5 and the t value is 4.

Am I right in saying the standard error is being used in two different distributions?

The first distribution is the distribution of estimated coefficients for the population. This distribution has a mean of 2 (the population coefficient estimate) with a standard error of 0.5.

The second distribution is the distribution of t values. This distribution has a mean of 0 with a standard error of 0.5.

• The problematic nature of the last statement becomes clear when you consider that first, distributions do not have standard errors: they have standard deviations; and second, if that's what you mean, any $t$ distribution with 2 or fewer degrees of freedom does not even have a standard deviation. – whuber May 23 '14 at 19:09

## 3 Answers

The t distribution has a mean of 0 as stated (unless using the non-central t, but that is rare in regression), but its standard deviation is based on the degrees of freedom and will not be 0.5 in this case. So the short answer to your second question is "No".

I don't think you have the 1st question correct either, but I don't fully understand your description. If we had the resources to take every possible sample of the given size from the population (possibly infinite) and find the slope/coefficient from each sample, then took the standard deviation of those estimates then we would have the standard deviation of the sampling distribution of the coefficient. Since we don't usually have the resources to take more than one sample, we can use theory, assumptions, and things we have learned from simulations along with the sample data to compute an estimate of the standard deviation of the sampling distribution of the coefficient. Since that is a bit of a mouthful we use the phrase "Standard Error of the Coefficient" as a short version of "estimated standard deviation of the sampling distribution of the coefficient". The standard error is our best estimate of how much the coefficient would change from sample to sample, it can therefore be used in hypothesis tests or confidence intervals to take into account random variation and help us make inference about the true value of the coefficient based on our estimate from the observed data.

What you know is that if $\beta$ has mean 0 then $\hat t=\dfrac{\hat\beta}{\hat\sigma_\beta}$ is a $t$ of student.

What you usually do is to compute the value $\hat t$ to know if it is possible that mean of $\beta$ is 0.

To answer to this question you use the t test

The standard error applies to any null hypothesis regarding the true value of the coefficient. Thus the distribution which has mean 0 and standard error 0.5 is the distribution of estimated coefficients under the null hypothesis that the true value of the coefficient is zero. You can have as many distributions as null hypotheses you wish to consider.

The coefficient estimate from regression on particular sample data should be thought of in relation to the distribution around a null hypothesis, not as itself the mean of a distribution.

• what about the standard error used in constructing confidence intervals? That is not a null hypothesis. The standard error is for the sampling distribution, not just for hypothesis testing. – Greg Snow May 23 '14 at 19:11
• @GregSnow The standard error can be used to construct a confidence interval around the estimated coefficient, but does that imply some kind of distribution with the estimated coefficient as its mean? It wouldn't be the sampling distribution of the coefficient, since only by chance will the mean of that distribution equal the estimated coefficient from particular sample data. – Adam Bailey May 23 '14 at 22:07