Poisson normal approximation for comparing means of count data

Question

I'm trying to compare the values of two counts from an A/B test, $c_1$ and $c_2$.

If I assume the data is Poisson distributed then the mean and variance are $\lambda$ and the standard error is $\frac {\lambda}{n}$. Since we only have one observation for each count the means and SE are just $c_1$ and $c_2$ for each. If $\lambda$ is sufficiently large, I can assume that the Poisson distribution approximates a normal distribution. I'm therefore creating a test statistic which is $\frac{c2-c1}{\sqrt{c1+c2}}$, and comparing this to 1.96 to check the significance of the difference.

Is this a valid procedure?

Answer 1

The generally accepted way of comparing 2 samples from Poisson distributions, to see if they have the same mean, is to note that the sum of 2 Poisson random variables $X_1$ and $X_2$ with means $\lambda_1$ and $\lambda_2$ is Poisson with mean $\lambda_1 + \lambda_2$; and the conditional distribution of $X_1$ given $X_1 + X_2 = n$ is $Binomial(n, \lambda_1 / (\lambda_1 + \lambda_2))$.

So you would compute the probability of observing $X_2 = c_2$ under that.

See http://www.biostat.jhsph.edu/~bcaffo/651_2006/files/lecture27.pdf