4
$\begingroup$

In linear regression from a dataset $\{ (x_i, y_i), i=1,\cdots,N \}$, I wonder what distributions are for the slope and for the intercept?

In Excel output, t-tests are used to tell whether the slope mean is zero and the intercept mean is zero. But t-test assumes the distributions of slope and of intercept are normal distributions. So I would like to see whether this is true.

$\endgroup$
3
$\begingroup$

(edited)

The slope and intercept t tests are distributed as central random variables when the 'true' slope and intercept are zero and the error terms are normal, independent and homoskedastic (there's probably a few assumptions I missed). They are distributed as non-central t random variables when the true values are non-zero (and the errors are normal, independent, etc). As pointed out in the comments, the regression coefficients are normally distributed under these assumptions.

Generally for large sample size, the (non-central) t is a very good approximation for the distribution of the regression test statistics. You can test the robustness of the approximation yourself by Monte Carlo simulation. Simply create random data (according to some model), perform linear regression, record the t-test statistics (as well as the slope and intercept), and repeat. Then look at the distribution of regression test statistics. If your generating model violates the assumptions of independence, homoskedasticity, normality, you will see how robust the approximation by t distribution is; you can also look at the robustness of the normal approximation for the regression coefficients.

$\endgroup$
  • 1
    $\begingroup$ I think this answer needs some qualification, because under one straightforward interpretation of the question it's obviously not correct. Because all the regression coefficients are linear combinations of the dependent values, this makes them linear combinations of normally distributed data, whence they have normal distributions, not Student-t distributions. Are you perhaps discussing estimates of the slope and intercept and distributions of test statistics rather than sampling distributions? $\endgroup$ – whuber Oct 12 '11 at 17:27
  • 1
    $\begingroup$ The estimated slope and intercept are normally distributed; the ratio of the estimates to their estimated standard errors would be t or non-central t. $\endgroup$ – Karl Oct 12 '11 at 17:29
  • $\begingroup$ @whuber. oops! right you are. $\endgroup$ – shabbychef Oct 12 '11 at 17:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.