What are models for count data where the variance is not related to the sample size?

Question

For modeling count data that arises as the sum of Bernoulli random variables or according to an occurrence process with constant rate in some interval, there are many techniques, like using a Binomial distribution, a Poisson distribution, a Poission-Binomial distribution, zero-inflated models to handle over-dispersion, and many others.

All of these models share the property that the variance of the outcome count variable increases as the number of observations increases. For example, if a Binomial distribution arises as the sum of $n$ iid Bernoulli variables with parameter $p$, then as the location parameter $np$ shifts to the right (gets larger) as a function of $n$, the variance $np(1 - p)$ also gets larger. Same with a Poisson where the variance and mean are equal.

Consider a counting problem where you do not expect this to happen. It could be that you believe the parameter $p$ decreases proportionally as $n$ increases.

It could be also be something similar to a zero-inflated model, where each time you increase $n$ to $n + 1$, it is overwhelmingly more likely that you'll add a new observation $i = n + 1$ for which the corresponding Bernoulli has $p_{n + 1} == 0$, and it will not contribute to the overall count or variance. Only a very small fraction will contribute, and in fact that fraction might go towards 0 as $n\to\infty$.

Another example could be some type of "follow the herd" phenomenon, where as more observations are included, the count distribution looks like a discrete distribution over a small set of outcomes, because almost all of the underlying Bernoulli random variables snap to having $p == 1$, but a select few "dissenters" remain possible of randomly contributing 1 or 0 to the count, and those dissenters express all of the source of variance.

Do there exist discrete count models like this? Specifically with the property that as $n\to\infty$, the variance either stays constant (and potentially is described by a totally different parameter) or converges to 0 (so that the distribution gets tighter as the sample size increases, rather than getting wider like Binomial or Poisson).

I could use a Normal distribution as an approximation, but even so, I would need a way to base the choice of the variance parameter on some underlying count model which could not be like Binomial or Poisson or Poisson-Binomial, where variance increases as sample size increases.

Answer 1

One attractive class of models is obtained by discretizing any non-negative random variable $X.$ Let $X$ be in a distribution family $\mathcal{F}.$ Let $\alpha\gt 0$ be a number and set

$$Y = \lfloor X/\alpha \rfloor.$$

This is the process of "binning" by assigning $X$ to one of the intervals $[0,\alpha),$ $[\alpha,2\alpha),$ and so on, beginning the counting at $0.$

Computing the probability functions of $Y$.

Let $F_X$ be the distribution function of $X,$ defined by $F_X(x)=\Pr(X\le x)$ for any real number $x.$ Then for any natural number $k,$ the definitions imply

$$F_Y(k) = \Pr(Y \le k) = \Pr(X/\alpha \lt k+1) = \Pr(X \lt (k+1)\alpha) = 1 - S_X((k+1)\alpha)$$

where $S_X$ is the survival function for $X,$

$$S_X(x) = \Pr(X \ge x).$$

(When $X$ is continuous at $x,$ $S_X(x) = 1-F_X(x).$)

This makes it almost as easy to work with $Y$ as with $X$ for analysis, estimation, and so on. Moreover, $Y$ has a probability mass function

$$p_Y(k) = \Pr(Y=k) = \Pr(Y\le k) - \Pr(Y\le k-1)= S_X(k\alpha) - S_X((k+1)\alpha).$$

Controlling the mean and variance of $Y.$

This binning approximately multiplies $X$ by $1/\alpha.$ Thus, approximately (with the error due only to the discretization), the $n^\text{th}$ (raw) moment of $Y,$ $\mu_Y^n,$ must be close to a scaled moment of $X:$

$$\mu_Y^n \approx \alpha^{-n}\mu_X^n.$$

In particular,

1. $\operatorname{Var}(Y) = \mu_Y^2 - (\mu_Y^1)^2 \approx \alpha^{-2}\operatorname{Var}(X).$

2. $E[Y] = \mu_Y^1 \approx \alpha^{-1}\mu_X = \alpha^{-1}E[X].$

Consequently, if $\operatorname{Var}(X) = f(E[X])$ for $X\in\mathcal{F},$ then

$$\operatorname{Var}(Y) \approx \alpha^{-2}f(\alpha E[Y]).$$

In particular, when $f$ is a power function $f(x) = x^p,$ then

$$\operatorname{Var}(Y) \approx \alpha^{-2}\left(\alpha^ E[Y]\right)^p = \alpha^{p-2}f(E[Y]).$$

When $\alpha \gt 1$ this means the variance of $Y$ tends to be relatively smaller, compared to its expectation, than $X.$

Examples

Finally, everything works even when $X$ is a count variable (the "underlying count model" of the question). For instance, $X$ could be a Poisson$(\lambda)$ variable, for which $\operatorname{Var}(X)=E[X]=\lambda.$ If you fix $\alpha,$ then $Y$ is still parameterized by $\lambda$ and

$$\operatorname{Var}(Y) \approx \alpha^{-2}\lambda = \alpha^{-1}\left(\alpha^{-1}\lambda\right) = \alpha^{-1}E[Y].$$

If you set $\alpha=\lambda^{1/2},$ then $Y$ is still parameterized by $\lambda$ but now the variance of $Y$ is approximately constant.

If you set $\alpha=\lambda,$ the expectation of $Y$ is approximately $1$ but its variance decreases asymptotically to $0$ as $\lambda$ grows.

(You can find exact, closed formulas for the moments of $Y$ when $X$ is Poisson by applying the techniques illustrated at https://stats.stackexchange.com/a/35138/919. )

These models all have simple, natural interpretations, especially when $\alpha$ is a whole number: you are still counting things (that's what $X$ does), but are grouping those things into bunches of size $\alpha$ (the last bunch will have somewhere between $1$ and $\alpha$ things in it) and reporting how many bunches there are (which is $Y+1$).