2
$\begingroup$

Recently I read Wald test for logistic regression this post, which states why do we use Wald test in logistic regression and no t-test instead. It says, that the main reason behind it, is that for example in OLS the variance of residuals is unknown, and it has to be estimated, so t-test has to be used. On the another hand, since logistic regression is estimated using maximum likelihood estimation, we know that $\hat \beta - \beta$ will be asymptotically normally distributed, and therefore the expression:

$$W = \frac{\hat \beta - \beta}{\hat{\textrm{se}}(\hat \beta)}$$

will have asymptotically standard normal distribution. Where of course:

$$\hat{\textrm{se}}(\hat \beta) = \sqrt{s^2(X^TX)^{-1}_{jj}}$$

and $s^2$ is a variance of residuals.

I understand, that in linear regression, you replace $s^2$ by its estimator $\hat \sigma^2$, and then statistics $W$ has t-student distribution instead of normal. However, when you fit exactly the variance, $W$ statistics will have asymptotically normal distribution. And here I couldn't find any information how exactly this variance in logistic regression can be found. Could you please explain to me how it can be dervied?

$\endgroup$

1 Answer 1

0
$\begingroup$

I have to agree that reading the post you linked and other sources I was confused as well. Digging around I was able to find this source which finally gives the full answer on how to calculate $SE(\hat{\beta_j})$. In summary:

$$\hat{SE}(\hat{\beta_j}) = \sqrt{(X^t\hat{W}X)_{jj}^{-1}}$$

Where $\hat{W}$ is a diagonal $nXn$ matrix:

$$\hat{W} = diag (\frac{e^{\Sigma_{j=0}^p\beta_jx_{1j}}}{(1+e^{\Sigma_{j=0}^p\beta_jx_{1j}})^2}, \dots, \frac{e^{\Sigma_{j=0}^p\beta_jx_{nj}}}{(1+e^{\Sigma_{j=0}^p\beta_jx_{nj}})^2})$$

From there you can go and stick it in the Wald statistic to get the t-value for the test of $H_0:\beta_j=0$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.