Why does Logistic Regression use the Wald test, whereas Linear Regression uses the t-test?

What causes the difference?


Below is a short explanation. If you have questions, I am happy to elaborate or go into further detail.

Linear Regression:

The use of t-tests is linear regression comes from the distribution of normally distributed error terms:

$y_i=X_i'\beta + \epsilon_i$

where $\epsilon_i \sim N(0,1)$ iid. It follows that

$\frac{\hat{\beta_j}-\beta_{j0}}{se(\hat{\beta_j})} \sim t(N-K),$

where $N$ is the sample size and $K$ is the length of the vector $\beta$.

Note that the default in most regression software packages test the hypothesis that $\hat{\beta_j}=0$, i.e. setting $\beta_{j0}$ equal to zero.

Logistic Regression:

Logistic regression assumes errors follow the logistic distribution. Consequently, the term $\frac{\hat{\beta_j}-\beta_{j0}}{se(\hat{\beta_j})}$ does not follow a t-distribution. Instead, we can use the Wald test, which relies on asymptotic normality as is implied by the Central Limit Theorem.

  • $\begingroup$ Logistic regression assumes errors follow the logistic distribution. Do you know of the derivation/explanation for this? $\endgroup$ – Trajan Apr 21 '20 at 11:10
  • 1
    $\begingroup$ Logistic regression is motivated by having a continuous, unobserved (latent) variable that determines the binary outcome: $y^*=X\beta+\epsilon.$ When the latent variable is positive (negative), the binary dependent variable takes a value of 1 (0). Logistic regression assumes $\epsilon_i$ follows standard logistic distribution, which is used to derive the likelihood function and estimate the model parameters $\beta.$ $\endgroup$ – dlnB Apr 21 '20 at 14:23
  • $\begingroup$ I can elaborate further if you wish. Please accept the answer if you are satisfied. $\endgroup$ – dlnB Apr 21 '20 at 14:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.