How can I test if given samples are taken from a Poisson distribution?

Question

I know of normality tests, but how do I test for "Poisson-ness"?

I have sample of ~1000 non-negative integers, which I suspect are taken from a Poisson distribution, and I would like to test that.

Answer 1

First of all my advice is you must refrain from trying out a Poisson distribution just as it is to the data. I suggest you must first make a theory as to why should Poisson distribution fit a particular dataset or a phenomenon.

Once you have established this, the next question is whether the distribution is homogeneous or not. This means whether all parts of the data are handled by the same poisson distribution or is there a variation in this based on some aspect like time or space. Once you have convinced of these aspects, try the following three tests:

1. likelihood ratio test using a chi square variable
2. use of conditional chi-square statistic; also called poisson dispersion test or variance test
3. use of the neyman-scott statistic, that is based on a variance stabilizing transformation of the poisson variable

search for these and you will find them easily on the net.

Answer 2

Here is a sequence of R commands that may be helpful. Feel free to comment or edit if you spot any mistakes.

set.seed(1)
x.poi<-rpois(n=200,lambda=2.5) # a vector of random variables from the Poisson distr.

hist(x.poi,main="Poisson distribution")

lambda.est <- mean(x.poi) ## estimate of parameter lambda
(tab.os<-table(x.poi)) ## table with empirical frequencies


freq.os<-vector()
for(i in 1: length(tab.os)) freq.os[i]<-tab.os[[i]]  ## vector of emprical frequencies

freq.ex<-(dpois(0:max(x.poi),lambda=lambda.est)*200) ## vector of fitted (expected) frequencies

acc <- mean(abs(freq.os-trunc(freq.ex))) ## absolute goodness of fit index acc
acc/mean(freq.os)*100 ## relative (percent) goodness of fit index

h <- hist(x.poi ,breaks=length(tab.os))
xhist <- c(min(h$breaks),h$breaks)
yhist <- c(0,h$density,0)
xfit <- min(x.poi):max(x.poi)
yfit <- dpois(xfit,lambda=lambda.est)
plot(xhist,yhist,type="s",ylim=c(0,max(yhist,yfit)), main="Poison density and histogram")
lines(xfit,yfit, col="red")

#Perform the chi-square goodness of fit test 
#In case of count data we can use goodfit() included in vcd package
library(vcd) ## loading vcd package
gf <- goodfit(x.poi,type= "poisson",method= "MinChisq")
summary(gf)
plot(gf,main="Count data vs Poisson distribution")

Answer 3

For a Poisson distribution, the mean equals the variance. If your sample mean is very different from your sample variance, you probably don't have Poisson data. The dispersion test also mentioned here is a formalization of that notion.

If your variance is much larger than your mean, as is commonly the case, you might want to try a negative binomial distribution next.

Answer 4

I suppose the easiest way is just to do a chi-squared Goodness of fit test.

In fact here's nice java applet that will do just that!

Answer 5

You can use the dispersion (ratio of variance to the mean) as a test statistic, since the Poisson should give a dispersion of 1. Here is a link to how to use it as a model test.

Answer 6

You can draw a single figure in which the observed and expected frequencies are drawn side by side. If the distributions are very different and you also have a variance-mean ratio bigger than one, then a good candidate is the negative binomial. Read the section Frequency Distributions from The R Book. It deals with a very similar problem.

Answer 7

Yet another way to test this is with a quantile quantile plot. In R, there is qqplot. This directly plots your values against a normal distribution with similar mean and sd

Answer 8

I think the main point is the one sidmaestro raises...does the experimental setup or data generation mechanism support the premise that the data might arise from a Poisson distribution.

I'm not a big fan of testing for distributional assumptions, since those tests typically aren't very useful. What seems more useful to me is to make distributional or model assumptions that are flexible and reasonably robust to deviations from the model, typically for purposes of inference. In my experience, it is not that common to see mean=variance, so often the negative binomial model seems more appropriate, and includes the Poisson as a special case.

Another point that is important in going for distributional testing, if that's what you want to do, is to make sure that there aren't strata involved which would make your observed distribution a mixture of other distributions. Individual stratum-specific distributions might appear Poisson, but the observed mixture might not be. An analogous situation from regression only assumes that the conditional distribution of Y|X is normally distributed, and not really the distribution of Y itself.