In the linear model $\mathbf{Y} = \mathbf{X}\beta + \epsilon$, where $\epsilon \sim N(0, \sigma^2 \mathbb{I})$, it is known the the standard error of the estimator $\hat{\beta}$ is given by

$$Var(\hat{\beta})=\sigma^2(\mathbf{X}^T\mathbf{X})^{-1}. $$

In one of the posts (How are the standard errors of coefficients calculated in a regression?), a representation was made as follows:

$$ \mathbf{X}= \begin{pmatrix}1 & x_1 \\ 1 & x_2 \\ \vdots & \vdots \end{pmatrix} \qquad \text{and} \qquad \beta = \begin{pmatrix} a \\ b \\ \vdots \end{pmatrix}$$

Question: Is $\mathbf{X}$ here just a simplifying assumption? If I multiply $\mathbf{X}\beta$ (plus the error term), then it becomes $y_i = a + bx_i + \epsilon_i$, which is nicely the linear regression model.

Question: With the result of $Var(\hat{\beta})$ above, how can I get the standard error for the slope and intercept estimates, $Var(\hat{\beta_0})$ and $Var(\hat{\beta_1})$? (Assuming $Y={\beta_0}+\beta_1X $).

Edit: In the link I gave, I was able to follow the derivation. I just didn't understand:

$$\sqrt{\widehat{\textrm{Var}}(\hat{b})} = \sqrt{[\hat{\sigma}^2 (\mathbf{X}^{\prime} \mathbf{X})^{-1}]_{22}} = \sqrt{\frac{n \hat{\sigma}^2}{n\sum x_i^2 - (\sum x_i)^2}}.$$

What is $[\cdots]_{22}$ inside the square root?

Your insights would be great.

  • 2
    $\begingroup$ You didn't successfully reproduce what was at the post, nor did you quite generalize it. If your $X$ only has 2 columns, $\beta$ can only have two elements (as at the linked post), and you'll be doing simple linear regression. Generally in multiple regression, $X$ will have $p+1$ columns (including the constant) and $\beta$ will have $p+1$ elements. $\endgroup$ – Glen_b -Reinstate Monica Apr 10 '15 at 5:50
  • $\begingroup$ The post you linked to itself links to discussion of the general case. $\endgroup$ – Glen_b -Reinstate Monica Apr 10 '15 at 5:55
  • 1
    $\begingroup$ $[]_{22}$ means the entry in the 2nd row and 2nd column of the matrix between the square brackets, i.e. the variance of $\hat{b}$ in this case. $\endgroup$ – ocram Apr 10 '15 at 6:00
  • $\begingroup$ Is there a reason why we take the second row, second column? (I am not familiar statistics using matrices.) $\endgroup$ – cgo Apr 10 '15 at 6:04
  • 1
    $\begingroup$ In a variance-covariance matrix of a vector, the variances are on the diagonal and covariances are off-diagonal. You want the variance of the second component of the vector, which will be the second element along the diagonal of the variance-covariance matrix, hence the 2,2. $\endgroup$ – Silverfish Apr 10 '15 at 6:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.