2
$\begingroup$

In the least square linear regression model, if the explanatory variables and the error term are independent, and the error term is normal, why does the t-statistic of $\hat{\beta}$ follow a t distribution? I understand that if x is nonrandom, then the t-statistic of $\hat{\beta}$ follow a t distribution.

In other words, $\hat{\beta}$ is Gaussian conditional on X. The unconditional distribution of $\hat{\beta}$ is non-Gaussian.

How can the test statistic still follow a t distribution?

$\endgroup$
3
  • 8
    $\begingroup$ Are you asking why, in the model $Y = X\beta + \varepsilon$ with $\varepsilon \sim \mathcal N(0, \sigma^2 I)$, we have $\hat \beta_j \sim t$? Because if so, that's not correct; $\hat \beta \sim \mathcal N(\beta, \sigma^2 (X^T X)^{-1})$. Maybe you're thinking of the test statistic? $\endgroup$
    – jld
    Jul 1, 2017 at 16:36
  • 1
    $\begingroup$ @Chaconne I believe that OP is asking: Why do we use a test statistic that follows a t-distribution (i.e., we use the t-statistic) to test $\hat{\beta}$? Why not, for instance, something that follows a normal distribution (i.e., z-score)? $\endgroup$
    – Mark White
    Jul 1, 2017 at 18:36
  • 2
    $\begingroup$ The estimate $\hat\beta$ doesn't follow a t distribution. Are you perhaps referring to the estimate divided by its estimated standard error? $\endgroup$
    – whuber
    Jul 2, 2017 at 14:01

2 Answers 2

3
$\begingroup$

The standardized $\hat{\beta}$ does have a t-distribution. While we usually are interested in that distribution conditional on the observed value of the regressor $X$, since that is what is relevant for statistical inference (see What are the differences between stochastic and fixed regressors in linear regression model? and What is the difference between conditioning on regressors vs. treating them as fixed?).

But, since this conditional distribution of the test statistic ($\hat{\beta}$ divided by its standard error) does not depend on $X$, the unconditional distribution is in fact the same.

$\endgroup$
2
$\begingroup$

In other words, $\hat{\beta}$ is Gaussian conditional on X. The unconditional distribution of $\hat{\beta}$ is non-Gaussian.

How can the test statistic still follow a t distribution?

$\hat{\beta}$ is Gaussian distributed (conditional on X), but the test statistic is $\frac{\hat{\beta}}{\hat{\sigma}_{\beta}}$. It follows a t-distribution because of the division with the estimated standard error.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.