I wonder about how the residuals of a logistic regression model should be distributed.

Of course, running a linear regression model and by assuming the Normal distribution assumption, the residuals you predicted from that kind of model should be distributed as a Normal distribution with mean $\mu$ $=$ $0$ and standard deviation $\sigma$ $=$ $1$;

But, what about if you run another kind of regression, with different distribution assumption as, for instance, the logistic one?

Let's suppose one runs a logistic regression model, what distribution the residuals should have?

And, moreover, what is the test I should run to check for the distribution assumption validity?

Any hint, reference or whatever will be appreciated.


Tweedie distributions are a family of probability distributions which include the purely continuous normal and gamma distributions, the purely discrete scaled Poisson distribution, and the class of mixed compound Poisson–gamma distributions which have positive mass at zero, but are otherwise continuous. For any random variable Y that obeys a Tweedie distribution, the variance var(Y) relates to the mean E(Y) by the power law,

$$\text{Var}(Y) = a[E(Y)]^p$$

where a is a scaling parameter and p the tail index parameter.

They include a number of distributions, each being specified by the domain of p:

  • normal distribution, p = 0,
  • Poisson distribution, p = 1,
  • compound Poisson–gamma distribution, 1 < p < 2,
  • gamma distribution, p = 2,
  • positive stable distributions, 2 < p < 3,
  • inverse Gaussian distribution, p = 3,
  • positive stable distributions, p > 3, and
  • extreme stable distributions, p = ∞

For 0 < p < 1 no Tweedie model exists.


Tail indexes can be estimated via standard metrics such as the Hill and Pickands esimators but Xavier Gabaix's heuristic using OLS regression and log-ranks is pretty straightforward and has the advantage of not requiring numerical integration.

See http://en.wikipedia.org/wiki/Tweedie_distribution for a general overview of Tweedies and Gabaix and Igragimov, RANK−1/2: A SIMPLE WAY TO IMPROVE THE OLS ESTIMATION OF TAIL EXPONENTS, 2009

All of that said and specifically wrt the issue of logistic regression residual diagnostics, I am aware of only one paper that treats this topic in any depth. It's by Daryl Pregibon and is titled simply Logistic Regression Diagnostics. https://projecteuclid.org/euclid.aos/1176345513

| cite | improve this answer | |
  • 2
    $\begingroup$ Thanks for the answer @MikeHunter, but my question was about the residuals distribution! For instance, what is the distribution of the residuals coming up the logistic distribution? Anyway, +1 because I did not know those things about distributions :) $\endgroup$ – Quantopik Jun 7 '15 at 13:23
  • 5
    $\begingroup$ The wording "with all due respect" doesn't cancel out "amazingly uninformed". I can't see that this answers the question either. I don't think saying that is being narrow or literal at all; your answer could be a good answer to a very different question, but not this one. $\endgroup$ – Nick Cox Jun 8 '15 at 12:04
  • 4
    $\begingroup$ @MikeHunter: What I'd like to do is keep the site well-organized & all contributors to it happy; it remains to be seen whether I have that power. Did you know you can ask & answer your own questions? - might be a good idea in this case, & would make your answer more prominent. (The Pregibon paper is an excellent choice by the way; a little more more on that would constitute a great answer by itself. But neither the OP nor I were asking anything about generalized extreme value distributions.) $\endgroup$ – Scortchi - Reinstate Monica Jun 8 '15 at 13:05
  • 3
    $\begingroup$ Mike, the first half of this post appears to have been copied wholesale from the Wikipedia article on Tweedie distributions without clear attribution. (Merely mentioning that article later does not suffice.) As such it misleads the reader by implicitly representing other peoples' work as your own. I have therefore edited your post to make it clear which part is from this article. Please see stats.stackexchange.com/help/referencing for our policy about referencing material. $\endgroup$ – whuber Jun 8 '15 at 13:15
  • 4
    $\begingroup$ Mike, I fear you might be confusing pedantry with a desire for clear honest communication. This site is not just about answering questions: it aims to curate the Q's and the A's both. For this to be possible and useful, it is necessary that all posts have a good chance of being understood reliably by all interested readers. I encourage you to interpret the comments and mod. interventions in this thread (and any other) in that light. Otherwise they will indeed seem to be mere pedantry, you could get discouraged, and we would (regretfully) lose someone with exceptional knowledge and experience. $\endgroup$ – whuber Jun 8 '15 at 18:56

Not the answer you're looking for? Browse other questions tagged or ask your own question.