For modeling count data that arises as the sum of Bernoulli random variables or according to an occurrence process with constant rate in some interval, there are many techniques, like using a Binomial distribution, a Poisson distribution, a Poission-Binomial distribution, zero-inflated models to handle over-dispersion, and many others.

All of these models share the property that the variance of the outcome count variable increases as the number of observations increases. For example, if a Binomial distribution arises as the sum of $n$ iid Bernoulli variables with parameter $p$, then as the location parameter $np$ shifts to the right (gets larger) as a function of $n$, the variance $np(1 - p)$ also gets larger. Same with a Poisson where the variance and mean are equal.

Consider a counting problem where you do not expect this to happen. It could be that you believe the parameter $p$ decreases proportionally as $n$ increases.

It could be also be something similar to a zero-inflated model, where each time you increase $n$ to $n + 1$, it is overwhelmingly more likely that you'll add a new observation $i = n + 1$ for which the corresponding Bernoulli has $p_{n + 1} == 0$, and it will not contribute to the overall count or variance. Only a very small fraction will contribute, and in fact that fraction might go towards 0 as $n\to\infty$.

Another example could be some type of "follow the herd" phenomenon, where as more observations are included, the count distribution looks like a discrete distribution over a small set of outcomes, because almost all of the underlying Bernoulli random variables snap to having $p == 1$, but a select few "dissenters" remain possible of randomly contributing 1 or 0 to the count, and those dissenters express all of the source of variance.

Do there exist discrete count models like this? Specifically with the property that as $n\to\infty$, the variance either stays constant (and potentially is described by a totally different parameter) or converges to 0 (so that the distribution gets tighter as the sample size increases, rather than getting wider like Binomial or Poisson).

I could use a Normal distribution as an approximation, but even so, I would need a way to base the choice of the variance parameter on some underlying count model which could not be like Binomial or Poisson or Poisson-Binomial, where variance increases as sample size increases.

  • $\begingroup$ the last sentence reads "use a Normal distribution..." are you referring to a delta method approximation? even in those cases the variance will depend on $n$ the sample size. $\endgroup$ Sep 27, 2018 at 17:09
  • $\begingroup$ No, I mean that if there are any count models exhibiting the variance property I am seeking (that variance is either unaffected by sample size, or might even be decreasing as a function of sample size), then I could potentially use a Normal approximation if the distributions are very tricky to work with and there was theoretical justification for the normal distribution. For example, in the "follow the herd" example above, this could work, because the mean shifts to the right as a function of sample size, but the variance is constant no matter what. $\endgroup$
    – ely
    Sep 27, 2018 at 17:44
  • $\begingroup$ The only problem in that case, is that the outcome is a discrete variable with a small number of possible outcomes centered on the mean and extended to the left or right. Sort of like taking a dice roll and adding N, so the tails of the Normal are too big a departure from the real distribution in that case. Imagine as N gets bigger, k people are given coins to flip, and the other $N-k$ become constants equal to 1. k could be a fixed constant, could grow as N grows, or could follow a weird quadratic where $k$ is low for low N, high for medium N, then low again for very high N. $\endgroup$
    – ely
    Sep 27, 2018 at 17:46
  • $\begingroup$ sounds like either a Yule-Simon: en.wikipedia.org/wiki/Yule%E2%80%93Simon_distribution, or a Zipf: en.wikipedia.org/wiki/Zipf%27s_law $\endgroup$ Sep 27, 2018 at 18:49
  • $\begingroup$ Those would work for modeling fast-decaying frequency counts, like in word counts or popularity counts. I am also seeking count models that produce more Normal-like distributions, where the mean is related to the sample size, but the variance is a separate parameter. You can shift a Normal's scale parameter (mean) to the right along the positive axis without affecting its mean. I'm looking for something that is similar, but describes count data where the variance does not end up being a sum or other increasing function of the sample size. $\endgroup$
    – ely
    Sep 27, 2018 at 19:34

1 Answer 1


One attractive class of models is obtained by discretizing any non-negative random variable $X.$ Let $X$ be in a distribution family $\mathcal{F}.$ Let $\alpha\gt 0$ be a number and set

$$Y = \lfloor X/\alpha \rfloor.$$

This is the process of "binning" by assigning $X$ to one of the intervals $[0,\alpha),$ $[\alpha,2\alpha),$ and so on, beginning the counting at $0.$

Computing the probability functions of $Y$.

Let $F_X$ be the distribution function of $X,$ defined by $F_X(x)=\Pr(X\le x)$ for any real number $x.$ Then for any natural number $k,$ the definitions imply

$$F_Y(k) = \Pr(Y \le k) = \Pr(X/\alpha \lt k+1) = \Pr(X \lt (k+1)\alpha) = 1 - S_X((k+1)\alpha)$$

where $S_X$ is the survival function for $X,$

$$S_X(x) = \Pr(X \ge x).$$

(When $X$ is continuous at $x,$ $S_X(x) = 1-F_X(x).$)

This makes it almost as easy to work with $Y$ as with $X$ for analysis, estimation, and so on. Moreover, $Y$ has a probability mass function

$$p_Y(k) = \Pr(Y=k) = \Pr(Y\le k) - \Pr(Y\le k-1)= S_X(k\alpha) - S_X((k+1)\alpha).$$

Controlling the mean and variance of $Y.$

This binning approximately multiplies $X$ by $1/\alpha.$ Thus, approximately (with the error due only to the discretization), the $n^\text{th}$ (raw) moment of $Y,$ $\mu_Y^n,$ must be close to a scaled moment of $X:$

$$\mu_Y^n \approx \alpha^{-n}\mu_X^n.$$

In particular,

  1. $\operatorname{Var}(Y) = \mu_Y^2 - (\mu_Y^1)^2 \approx \alpha^{-2}\operatorname{Var}(X).$

  2. $E[Y] = \mu_Y^1 \approx \alpha^{-1}\mu_X = \alpha^{-1}E[X].$

Consequently, if $\operatorname{Var}(X) = f(E[X])$ for $X\in\mathcal{F},$ then

$$\operatorname{Var}(Y) \approx \alpha^{-2}f(\alpha E[Y]).$$

In particular, when $f$ is a power function $f(x) = x^p,$ then

$$\operatorname{Var}(Y) \approx \alpha^{-2}\left(\alpha^ E[Y]\right)^p = \alpha^{p-2}f(E[Y]).$$

When $\alpha \gt 1$ this means the variance of $Y$ tends to be relatively smaller, compared to its expectation, than $X.$


Finally, everything works even when $X$ is a count variable (the "underlying count model" of the question). For instance, $X$ could be a Poisson$(\lambda)$ variable, for which $\operatorname{Var}(X)=E[X]=\lambda.$ If you fix $\alpha,$ then $Y$ is still parameterized by $\lambda$ and

$$\operatorname{Var}(Y) \approx \alpha^{-2}\lambda = \alpha^{-1}\left(\alpha^{-1}\lambda\right) = \alpha^{-1}E[Y].$$

If you set $\alpha=\lambda^{1/2},$ then $Y$ is still parameterized by $\lambda$ but now the variance of $Y$ is approximately constant.

If you set $\alpha=\lambda,$ the expectation of $Y$ is approximately $1$ but its variance decreases asymptotically to $0$ as $\lambda$ grows.

(You can find exact, closed formulas for the moments of $Y$ when $X$ is Poisson by applying the techniques illustrated at https://stats.stackexchange.com/a/35138/919. )

These models all have simple, natural interpretations, especially when $\alpha$ is a whole number: you are still counting things (that's what $X$ does), but are grouping those things into bunches of size $\alpha$ (the last bunch will have somewhere between $1$ and $\alpha$ things in it) and reporting how many bunches there are (which is $Y+1$).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.