8
$\begingroup$

Suppose we have a logistic regression model:

$$\begin{align} P(y=1\vert\mathbf{x}) &= p \\ \log\left(\frac{p}{1-p}\right) &= \boldsymbol{\beta}\mathbf{x} \end{align}$$

Given a random sample $D=\{\mathbf{X},\mathbf{y}\}$ of size $N$, we can compute confidence intervals for the $\boldsymbol{\beta}$ and correspondingly prediction intervals for $p$, given a certain value $\mathbf{x}^*$ of the predictor vector. This is all very standard and detailed, for example, here.

Suppose instead that I'm interested in a prediction interval for $y$, given $\mathbf{x}^*$. Of course, it doesn't make any sense at all to compute a prediction interval for a single realization of $y$, because $y$ can only take the values 0 and 1, and no value in between. However, if we consider $m$ realizations of $y$ for the same fixed value of $\mathbf{x}^*$ , then this becomes similar (but not identical) to the question of computing a prediction interval for a binomial random variable. This is basically the same situation described by Glen_b in the comments to this answer. Does this question have an answer, apart from the trivial one "use nonparametric bootstrap"?

$\endgroup$
4
  • $\begingroup$ can you compute a prediction interval for $log(p / (1-p))$ instead perhaps? $\endgroup$ Mar 31 '18 at 1:15
  • 2
    $\begingroup$ @HughPerkins I think that the issue is how to combine the uncertainty in p with the uncertainty in binomial sampling also given the uncertainty in p. Is there a closed-form solution? $\endgroup$
    – EdM
    Mar 31 '18 at 2:17
  • $\begingroup$ @EdM you got my point. I wonder if there is a closed form solution or an analytical approximation. $\endgroup$
    – DeltaIV
    Mar 31 '18 at 12:03
  • 1
    $\begingroup$ [offtopic] random idea, it occurs to me that it could be interesting to have a tag like 'open-research-opportunity' for questions like this which/if they are answered in the negative $\endgroup$ Mar 31 '18 at 16:05
4
$\begingroup$

One way this should work without bootstrapping (which in practice may be the fastest thing tho implement), would be:

  1. Assume that a normal approximation for the predicted log-odds ($x \hat{\beta}$) plus/minus its standard error works. Any logistic regression software will provide this.
  2. The percentiles of this distribution transform to probabilities via the anti-logit.
  3. One can find a (mixture of) beta distribution(s) that approximates the predictive distribution for the probability well.
  4. The predictive distribution for the outcome is then a (mixture of) beta-binomial distribution(s with the same mixing weights as used in step 3).

Alternatively, one can "just" integrate out the log-odds from the joint predictive of outcome and log-odds, but I believe that will be a complete mess with no closed form solution.

$\endgroup$
12
  • 4
    $\begingroup$ You could also just directly simulate from the asymptotic multivariate normal for $\beta-\hat{\beta}$, and then form a mixture of binomials over those values. $\endgroup$
    – Glen_b
    Apr 1 '18 at 6:28
  • $\begingroup$ I like the overall idea, but I'm not sure about the details. For example, "find a (mixture of) beta distribution(s) that approximates the predictive distribution for the probability well", How do you this in practice? Could you add an example? Even a low-dimensional one would suffice. $\endgroup$
    – DeltaIV
    Apr 2 '18 at 6:07
  • 3
    $\begingroup$ I can write this up as something in the form of an answer if you prefer -- I don't mind either way. $\endgroup$
    – Glen_b
    Apr 3 '18 at 14:58
  • 2
    $\begingroup$ @Glen_b I'd really appreciate that. $\endgroup$
    – DeltaIV
    Apr 6 '18 at 18:25
  • 1
    $\begingroup$ @Glen_b, I would be interested in seeing that answer. $\endgroup$ Jun 24 '19 at 8:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.