2
$\begingroup$

I'm learning linear regression. I'm stuck on how to calculate the standard error of the coefficients. I know that in the simple linear regression provided by this equation $ y = \beta_0 + \beta_1x + \varepsilon$ is:
\begin{align} SE(\hat\beta_0)^2 &= \sigma^2 \left( \frac{1}{n} + \frac{\bar{x}^2}{\sum^{i = 1}_{n} (x_i - \bar{x})^2} \right ) \\[5pt] SE(\hat\beta_1)^2 &= \frac{\sigma^2}{\sum^{i = 1}_n (x_i - \bar{x} ) ^2} \end{align}

Given that $\sigma^2$ is the variance of the error term ($\,\varepsilon\,$)

But when the model is extended to $k$ variables ($y = \beta_0 + \beta_1x_1 + \beta_2x_2 + ... + \beta_kx_k + \varepsilon$), can we have a general formula to calculate each $SE(\hat\beta_i)$ (given that all the model assumption about the data is correct)?

$\endgroup$

1 Answer 1

3
$\begingroup$

To get a general formula for a multiple regression coefficient standard error, you need to use linear (matrix) algebra. The formula for the variance-covariance matrix of the betas is:
$$ VCOV(\hat{\beta}) = s^2{\bf (X'X)^{-1}} $$ From there, you take the positive square root of the $j^{\rm th}$ diagonal element to get the standard error of that beta.

$\endgroup$
12
  • 3
    $\begingroup$ @Miku, if X is not invertible, then the model cannot be fit, so you won't have to worry about the SE. $\endgroup$ Commented Sep 2, 2021 at 0:47
  • 1
    $\begingroup$ @Miku, if it's not invertible, you won't have a model. If you have a model, then the matrix was invertible. Inverting the matrix is part of the process of fitting the model. $\endgroup$ Commented Sep 2, 2021 at 1:41
  • 1
    $\begingroup$ Ohh thank you, I got it now :D $\endgroup$
    – Miku
    Commented Sep 2, 2021 at 1:45
  • 1
    $\begingroup$ You're welcome, @Miku. $\endgroup$ Commented Sep 2, 2021 at 2:34
  • 1
    $\begingroup$ @Miku $\hat\beta = (X'X)^{-1}X'y$, which is why it must be that $X'X$ is invertible. $\endgroup$
    – Dave
    Commented Sep 8, 2021 at 21:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.