Estimate Poisson mean with partial data?

Question

I would like to estimate Poisson means from data sets where means are usually in the range 0.5 to 10. 0 (zeros) in the data are unambiguous; estimates of higher-event numbers degrade in rough proportion to the number of events.

This R code is a reasonable data simulator:

data<-rpois(100, 1.5)
datasim <- function (x, upto = 6) {
 counts <- matrix(nrow = (upto+1), ncol = 2)
 for (i in 1:(upto+1)) {
    counts[i,1] <- i-1
    counts[i,2] <- length(which(x==i-1))
 }
return(counts)
}
simdata<-datasim(data, upto=3)

> simdata
     [,1] [,2]
[1,]    0   19
[2,]    1   42
[3,]    2   19
[4,]    3   10

In general: if I have a subset of event counts, how can I estimate the Poisson mean and CI? I've looked at 'fitdistr', glm's, Poisson.exact, etc. but I haven't found a solution to this problem - I'd like to assure myself that the solution explicitly recognizes that some events are excluded from the data, and that the number of excluded data are known - that n=100 in the example.

Answer 1

For a censored Poisson you can use the UWSE. In particular, for your example, the vector of sets $ (\left\{0\right\},\left\{1\right\},\left\{2\right\},\left\{3\right\}) $ generates a unique weight vector. So, according to the explanation here: https://paradsp.wordpress.com/ (bottom of site), a consistent estimate is:

$$ \hat{\lambda_{UWSE}} = \min_{\lambda > 0}||(e^{-\lambda},\lambda e^{-\lambda},\frac{\lambda^2e^{-\lambda}}{2},\frac{\lambda^3e^{-\lambda}}{6} ) - (\hat{w_{0}},\hat{w_{1}} ,\hat{w_{2}},\hat{w_{3}})||_2 $$

where $\hat{w_{0}},\hat{w_{1}} ,\hat{w_{2}},\hat{w_{3}} $ are the relative frequencies of the obervations $ 0,1,2,3 $ respectively. For the data given in the example: $\hat{w_{0}}= 0.19,\hat{w_{1}} = 0.42 ,\hat{w_{2}} = 0.19,\hat{w_{3}} = 0.10 $. Computing this numerically in R, I get:

$$ \hat{\lambda_{UWSE}} = 1.47$$

which is quite close to the true value of $ 1.5 $.

If you want a quick closed form solution ,but trading off for some accuracy, choosing the set $\left\{0\right\} $ is enough to generate a unique weight. Then:

$$ \hat{\lambda_{UWSE}} = -ln(\hat{w_{0}}) = 1.66 $$

which is a bit further off.