# Why use the Wald test in logistic regression?

Some statistical software use the Wald statistic when reporting on the regression coefficients. As examples, R and Stata report Wald by default.

The logistic regression article on Wikipedia says, unfortunately without reference:

“Rather than the Wald method, the recommended method to calculate the p-value for logistic regression is the likelihood-ratio test (LRT)”

How are Wald and LRT calculated for a logistic regression coefficient (independent variable)? This is for a reminder on how they are calculated and to highlight their differences.

From the Wald test page on Wikipedia:

• [T]he Wald test is not invariant to a reparametrization, while the Likelihood ratio tests will give exactly the same answer whether we work with R, log R or any other monotonic transformation of R.

So in the context of logistic regression, if you logged a regressor its p-value would be different compared to if it was unlogged (is this correct). Would the p-value change if the LRT was used?

From the same Wald test page:

• The other reason is that the Wald test uses two approximations (that we know the standard error, and that the distribution is χ2), whereas the likelihood ratio test uses one approximation (that the distribution is χ2).

Although Wald and likelihood ratio are asymptotically equivalent, in the logistic regression we are usually in the pre-asymptote setting, so this is not a reason to view then as equivalent.

Thus it seems that the Wald test disadvantages outweigh the advantages in the logistic setting, and the likelihood ratio is better.

It is my guess that the Wald test is used by logistic regression software routines for its easier computational efficiency, which was more important in the past when software such as R and Stata were first created. Then, through backwards compatibility and not wanting to change the semantics of their logistic functions, the Wald statistic has remained the default. Is there any evidence for this being the case?

Should I be changing the default Wald to likelihood ratio? A lesser question, is there an easy was to do this in R?

• You have the covariance matrix for the coefficient estimates to hand after fitting a model, so performing Wald tests on $p$ coefficients is just a matter of taking $p$ square roots. Performing the corresponding likelihood-ratio tests, on the other hand, would require fitting $p$ new models. Given that the analyst mightn't be interested in performing any tests at all, it's probably for the best that software displays the results Wald tests in default model aummaries. Mar 7 at 0:49
• @Scortchi-ReinstateMonica this reasons is an excellent one for why Wald tests are the default. It could be the core part of an answer. I know that the Wald test and logged LRT measure respectively the horizontal and vertical distance between the maximum likelihood and null hypothesis value, but would like a clarification and context of the main differences they give you in the logistic regression setting, one of which would be how the Wald p-value is already nearly pre-calculated from the covariance matrix. Mar 7 at 8:16

In logistic regression (& other generalized linear models with canonical link functions), the coefficient estimates $$\hat\theta$$ are arrived at by Fisher Scoring: iterating $$\vec\theta_{k+1} = \vec\theta_k + \mathcal{I}^{-1}(\vec\theta_k)U(\vec\theta_k)$$ where $$\mathcal{I}$$ is the Fisher information & $$U$$ the score, until convergence. When you're done, you're left with the covariance matrix $$\mathcal{I}^{-1}$$ for the coefficient estimates; the square roots of its diagonal elements are the variances you need for Wald tests of each coefficient. So you get Wald tests for free, almost, just by fitting a model; but likelihood-ratio tests require fitting a new model for each coefficient you want to test—with a large sample size & many predictors they'd take a good while longer to conduct. (This is also true more generally: if you're using observed information (the negative Hessian of the log likelihood) rather than expected information; or even if you're finding maximum likelihood estimates with an algorithm that doesn't involve calculating the Hessian, it's quicker to evaluate the Hessian numerically than to fit lots of models.)
† I don't know of any R function to conduct LRTs for all coefficients of a model individually—it wouldn't be hard to write one—but both stats:::drop1 & car:::Anova conduct them for a default set of null hypotheses more likely to be of interest.
NB invariance to reparametrization means only that the LRT for, say, $$H_0: \beta_7 =0$$ is the same as the LRT for $$H_0: \frac{1}{1+\mathrm{e}^{-\beta_7}}=1$$ (which isn't the case for the Wald test). Replacing $$\beta_7$$ with $$\log \beta_7$$, on the other hand, would be fitting a substantively different model.