I have a data set where I am counting how many events happen ($N$) over a given duration ($t$), and am defining an event rate as $R=N/t$. What I'm wondering is what $N$ gives me a certain confidence on the rate. Looking on the web I found this formula:

$$ \Delta R = Z\sqrt{\frac{R}{t}}, $$ or $$ N = (\frac{t\Delta R}{Z})^2 $$

But, if this is the $N$ required to get a certain confidence interval on my rate, this equation means the smaller I want my confidence interval, the smaller $N$ becomes. This seems counter-intuitive, I feel like I have more confidence when $N$ is large.

Eg. if I am counting people walking by, and I count 3 in 60 minutes, that is 1/20 per minute and I don't have much confidence in that number. If, however, I saw 300 people in 60 minutes, then I am more confident that they come by at 5 per minute.

  • $\begingroup$ Your intuition is concerned with the relative magnitude of the interval width, which is proportional to the ratio of $\sqrt N$ to $N:$ indeed, that does get smaller as $N$ increases. $\endgroup$ – whuber Jun 21 '18 at 1:18
  • $\begingroup$ You didn't explain your symbol $Z$. $\endgroup$ – kjetil b halvorsen May 5 '19 at 12:33

(You didn't explain $Z$, please do so.) If we assume a Poisson distribution for your count $N_i$ (is is unclear if you have one or more counts) then $N_i \sim \mathcal{Po}(\lambda t_i)$ and the observed rate $R_i=N_i/t_i$ will have expectation $\lambda$, the parameter representing the rate. The standard deviation of $R_i$ will be $\sqrt{\lambda/t_i}$, and that could be used for an approximate confidence interval. $N$ is the count, an observation, you cannot choose it. If the rate $\lambda$ is large, then since expectation and variance in the Poisson distribution are equal, the standard deviation of the rate estimate becomes larger. What you can choose is the observation interval $t_i$, and by making that larger you can reduce the standard deviation.

It could be better to use the loglikelihood function and construct a CI directly from it, inverting the likelihood ratio test (This is effectively what is done in the R example below.)

If you have $n$ independent observations, say counts on different days, you could use Poisson regression, with the (log of) the observation time $t_i$ as an offset. In R code this could look like, using simulated data:

t <- c(1, 2, 3, 3, 3, 2, 5, 2, 3, 8)
lambda <- 3
N <- rpois(length(t), lambda*t)
yourdf <- data.frame(t, N)
mod <- glm(N ~ offset(log(t)), family=poisson, data=yourdf)

Call:  glm(formula = N ~ offset(log(t)), family = poisson, data = yourdf)


Note that $e^{1.159} \approx 3.19$ close to the value used in simulation $\lambda=3$. The we can get a confidence interval on $\lambda$ by

Waiting for profiling to be done...
   2.5 %   97.5 % 
2.608252 3.846705 

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.