2
$\begingroup$

I have fit my own logistic model and I am using it to calculate the probabilities of different outcomes. I would like to know how I can obtain the confidence intervals of these probabilities.

I am using scikit in python, but I am happy to know just the mathematical answer.

$\endgroup$
2
$\begingroup$

The typical logistic regression model is written as something like $$\log\left(\frac{\pi}{1-\pi}\right)=x^T\beta$$ where we model the log-odds by a linear combination of our predictor variables $x$. In the equation above $\pi$ would be the probability that you are interested in calculating a confidence interval for.

Now, rearranging terms, we know that we can estimate the probability $\pi$ as $$\hat\pi=\frac{e^{x^T\hat\beta}}{1+x^T\hat\beta}$$

where $\hat\beta$ are the estimated coefficents from your linear regression.

It should be noted that, since maximum likelihood estimates are invariant to transformation, $\hat\pi$ may also be considered the maximum likelihood estimate of $\pi$.

So now, construction of confidence interval proceeds using the fact that $$\frac{x^T\hat\beta-x^T\beta}{\hat{SE}}\stackrel{.}{\sim} z$$ where $$\hat{SE}=\sqrt{x^T(X^TWX)^{-1}x}$$

We can then construct a $(1-\alpha)$ confidence interval for $x^T\beta$ as $$(L,U)=\left(x^T\hat\beta-z_{\alpha/2}\hat{SE},\,x^T\hat\beta+z_{\alpha/2}\hat{SE}\right)$$ and thus, finally, a $(1-\alpha)$ confidence interval for $\pi$ is therefore $$\left(\frac{e^L}{1+e^L},\,\frac{e^U}{1+e^U}\right)$$


Some of the details above I have excluded assuming that you understand how a logistic regression works in general, and its corresponding design matrix, etc.

$\endgroup$
  • 1
    $\begingroup$ thanks for your answer! My only doubt in your answer is in the first formula where z appears. Is that z the z-score? Can you explain it a bit further? And what is the W of the next formula? Thanks again $\endgroup$ – Escachator Nov 10 '15 at 8:24
  • $\begingroup$ @Escachator that is the z-score. The distribution is approximately normal. $\endgroup$ – user30490 Nov 10 '15 at 15:12
  • $\begingroup$ I understand that the sigmoid function is the cdf of the normal distribution, however, in this case the sigmoid function is the distribution itself, i.e. is the pdf. That is my first doubt. The other is not sure how to understand that concrete formula ((xTβ^−xTβ) / SE^)... Thanks! $\endgroup$ – Escachator Nov 10 '15 at 16:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.