1
$\begingroup$

When we test the significance of a particular coefficient in linear regression, we assume that the sampling distribution of this coefficient is normally distributed given that the residuals of the regression model is normally distributed following the central limit theorem. As such, we obtain the p-value for the coefficient from the standard normal distribution. My first question is whether my understanding is correct and if yes, my second question is if there is an equivalent way of explaining how we go about testing the significance of a coefficent in logistic regression (i.e., assumption, sampling distribution).

$\endgroup$
1
$\begingroup$

For the linear regression, rather than the residuals, one usually assumes that the errors (the random unseen fluctuations in the response) are normal (and independent). Then, the residuals, and the estimated regression coefficients, will have normal sampling distributions as a consequence, and the inference machinery works exactly, not asymptotically. That is, there is no need to use the central limit theorem. If you don't want to start with the normality assumption for the errors, it is a different story.

For logistic regression, the problem is not as straightforward. Inference is usually based on the maximum likelihood estimator (MLE) and its asymptotic normality. The standard errors are estimated using the (empirical) Fisher information (the negative of the second derivative of the likelihood) which is an approximation to the inverse of the asymptotic variance of the MLE. Roughly speaking, asymptotically, you can expand the likelihood near the MLE using a Taylor expansion so that locally, it looks like you are fitting a linear model.

In the case of logistic regression, doing this expansion, the problem looks like a weighted linear regression, though with the weights dependent on the MLE, so the resemblance is only local and valid only in a neighborhood of the MLE. This link has some more details.

There are some consequences to all of this. For example, one uses the normal distribution to set the length of the confidence interval in logistic regression (or more generally the generalized linear models) rather than a $t$ distribution in the linear regression, and everything is approximate not exact.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.