0
$\begingroup$
  1. $Z=\frac{\hat{\beta}-0}{se(\hat{\beta})} \sim\ \mathcal{N}(0,1)$ where $H_0: \beta=0$
  2. $\hat{\beta} \sim\ \mathcal{N}(\beta, (\sigma_u)^2(X'X)^{-1})$ where $\frac{plim(X'X)}{n} = \sigma_{xx}$

I reckon that in equation (2), the latter is the covariance matrix for the variable $X$, and the former is a kind of variance concept applied to matrices; i.e., the average of the square of a variable, since the square of the average is zero(?) hence not included? Is this correct reasoning?

$\endgroup$

1 Answer 1

0
$\begingroup$

The first equation is the distribution of a single parameter of $\hat{\beta}$, say for example $\hat{\beta}_j$, not for the entire parameter vector $\beta$.

The second equation is the distribution of the entire parameter vector $\hat{\beta}$. $\sigma_u^2$ is the variance of the residuals in the "true" model (can be estimated using a RSS-type formula), while $\sigma_{xx}$ is the covariance matrix of $X$ in the training set distribution (estimated by $X^{\prime}X / n$).

Notice that in simple linear regression, (2) reduces to (1). Both formulas depend on the assumption that the residuals $u$ are Gaussian.

Ref: http://web.stanford.edu/~hastie/ElemStatLearn/ (Ch 3; free book).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.