Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I have read about the fact that, there is dependency of variance on mean of count data.In most of cases they do variance stabilization transfomration as preprocessing step of data modeling.
I wonder, why variance depends on mean in count based data ? in other word, which properties of count data make this happen ?
I propose that substantial insight into this question is afforded by viewing counts as sums of simple (happened vs. did not happen) events. That suffices to create a relationship between variance and expectation which in common situations amounts to a direct proportion.
Most counts are obtained in a context where numerous events could or could not have happened; the counts sum the events that did happen. By definition, such an event $i$ has a Bernoulli distribution: it had a chance of $p_i$ of occurring and, therefore, a chance of $1-p_i$ of not occurring. Their counts, therefore, are sums of Bernoulli variables.
The expectation of a sum is always the sum of the expectations. Thus, the expectation of $n$ Bernoulli variables with probabilities $p_i, i=1, 2,\ldots, n$ is the sum
$$p = \sum_{i=1}^n p_i.$$
When those variables are independent (and being "nearly" independent would be close enough), the variance of their sum is the sum of their variances. Since the variance of a Bernoulli$(p_i)$ variable is $p_i(1-p_i)$ (which is readily established from first principles), the variance of the sum is approximately
$$v = \sum_{i=1}^n p_i(1-p_i).$$
Although this is too complicated to allow any really general statements, we can make some useful deductions for common situations.
(Binomial sampling). When all the $p_i$ are equal, $p = np_1$ and $v = np_1(1-p_1)$. This exhibits $v$ as directly proportional to the expected count since
$$v = p(1-p_1).$$
(Poisson distribution). When $n$ is large and all the $p_i$ are so small that every $np_i$ is also small (say, less than $1$), then the $1-p_i$ terms in the general expression of $v$ are so close to $1$ as to be negligible, even when accumulated in the summations. Accordingly, to a good approximation,
$$v \approx \sum_{i=1}^n n p_i = p.$$
Again the variance is proportional to the expected count, but with a universal constant of proportionality equal to $1$.
First of all, it's not necessary to transform count data because there are Poisson and Negative Binomial models that allow the variance to depend on the mean.
Second, the dependency of variance on mean is not restricted to count data. I think it happens because the variation among experimental units happens on a "relative" rather than "absolute" scale.
E.g., consider a group of individuals whose average income is \$200,000. It's quite possible that there is one person in the group whose income is "relatively" low. Say, it's 80% of the mean, \$160,000.
Now consider another group where the average income is \$35,000. Again it's quite possible to have one person in the group whose income is 80% of the mean (\$28,000), but it's unlikely to see the one who is \$40,000 below the mean (-\$5000).
If the variance were independent of the mean, then observing \$160,000 income in the first group would be just as plausible as observing -\$5000 in the second group, but that's not the case.
