1
$\begingroup$

Let's keep it simple and just go with Simple Logistic Regression where there is only 1 continuous independent variable.

I.e. Log(P/1-P)=B0 + B1X1

What is the working out to derive the standard error for the B1 coefficient? Or what does the formula look like? I would also like to see the solution for this without getting linear algebra and matrices involved.

I'm guessing it wouldn't look anything like the formula for the standard error for B1 coefficient in simple linear regression i.e. √(s^2 / Sum of (Xi - Xbar)^2) or would it? Thank you

$\endgroup$

1 Answer 1

1
$\begingroup$

In logistic regression, as with most nonlinear models, you need to rely on asymptotic normality to characterize the sampling distribution of the coefficient. The variance of this distribution is given by the Fisher information of the logit log-likelihood. More formally, $$ \sqrt{n}(\hat{\beta}-\beta_0) \to N(0, \mathbf{A}^{-1}) $$ where $\mathbf{A}$ is the negative Hessian of the log-likelihood, and given by $$ \mathbf{A} = \lim_{n \to \infty} - \frac{1}{n} \operatorname{E} \left. \frac{\partial^2 \log L}{\partial \beta \partial \beta^\mathsf{T}} \right|_{\beta_0} \approx \frac{1}{n} \sum_{i=1}^n \Lambda_{i}(1-\Lambda_{i}) x_i x_i^\mathsf{T} $$ where $\Lambda_{i} = \frac{1}{1 + e^{-x_i\beta}}$ is the logistic function. For a more rigorous derivation and all the necessary assumptions behind this result, see Gourieroux & Monfort (1981).

$\endgroup$
2
  • $\begingroup$ Thank you. This is most definitely beyond my mathematical capabilities/understanding as I am not actually a statistics student. Would you have any sources that would be able to explain or introduce these concepts intuitively/conceptually rather than purely mathematically? Thank you. $\endgroup$
    – Mandem
    Commented Apr 17 at 11:44
  • $\begingroup$ A gentle heuristic derivation of this result specifically can be found in Appendix 17A&B of Wooldridge. If you are asking why the Fisher information serves as asymptotic sampling precision of the MLE, have a look at this question and the references therein. $\endgroup$
    – Durden
    Commented Apr 17 at 15:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.