4
$\begingroup$

There are five different periods (one following each other) of unequal lengths where one specific event is happening such that it follows Poisson distributions (e.g. meteorite falls). It is always the same type of event, just the expected average time between event is changing from time to time. In reality there are many more such periods.

The data look like this:

  • in the 1st period (100 hours) the average interval for event is 500 hours
  • in the 2nd period ( 80 hours) the average interval for event is 700 hours
  • in the 3rd period (130 hours) the average interval for event is 300 hours
  • in the 4th period ( 50 hours) the average interval for event is 800 hours
  • in the 5th period (400 hours) the average interval for event is 900 hours

Lambda:

  • 1st period lambda: (1/500)*100 = 0.200
  • 2nd period lambda: (1/700)*80 = 0.114
  • 3rd period lambda: (1/300)*130 = 0.433
  • 4th period lambda: (1/800)*50 = 0.062
  • 5th period lambda: (1/900)*400 = 0.444

Is there a way to estimate how many events happened and what is the confidence interval for the estimate?

I understood from How to calculate a confidence level for a Poisson distribution? and here: http://ms.mcmaster.ca/peter/s743/poissonalpha.html that I can use (x is number of events):

  • ( qchisq(0.025, 2*x)/2, qchisq(0.975, 2*(x+1))/2 )

or maybe some other method that will be more suitable for low lambda.

I don't understand how I can add all the five distributions together to say how many events were expected.

$\endgroup$
  • 1
    $\begingroup$ A confidence interval is constructed from data, but you don't mention data at all. It sounds like you just want to compute the distribution of the total number of events. Is that correct? If not, could you describe the data you have? It's also curious that you know what the Poisson intensities are. Do you truly know them or are you estimating them from some data? If so, what data? $\endgroup$ – whuber Oct 17 '19 at 10:35
  • $\begingroup$ @whuber You are correct. I want to know the expected distribution of the total number of events across these 5 periods. This exercise comes from mining of cryptocurrencies. When you mine, you do know how much time on average it will take you to find a coin given computational resources you have. At discrete non-fixed intervals (typically few weeks) the difficulty of mining will change and will alter your average time you expect to mine a coin. I am trying to aggregate the expectations over more periods of difficulty, add CI and eventually compare with reality. $\endgroup$ – Tomas Greif Oct 17 '19 at 12:50
2
$\begingroup$

Because (a) the number of events in any $T$ hour period with a rate of one event every $H$ hours follows a Poisson distribution with parameter $T/H$ and (b) the number of independent Poisson random variables of rates $\mu_1,\mu_2,\ldots,\mu_n$ in non-overlapping intervals is a Poisson random variable of rate $\mu=\mu_1+\mu_2+\cdots+\mu_n,$ any question about the probability distribution of the number of events is readily answered. Let's write $F$ for this distribution function and (therefore) $F^{-1}$ for its quantile function.

For instance,

  • The expected number of events is $\mu.$

  • The variance of the expectation is $\mu$ and so its standard deviation is $\sqrt{\mu}.$

  • For any tolerance level $q$ between $0$ and $100%,$ choose nonnegative values $\alpha_l$ and $\alpha_u$ totaling $1-q.$ Then with at least probability $q$ the number of events will be between $F_{l} =F^{-1}(\alpha_l)$ and $F_{u}=F^{-1}(1-\alpha_u).$ The actual probability is $F(F_{u}) - F(F_{l}-1).$

Often one chooses a "symmetrical" interval in the sense that $\alpha_l=\alpha_u = (1-q)/2.$

To see how simple this is, let's use the data in the question as an example.

  1. The time intervals are $100,80,130,50,400$ minutes and the rates are the reciprocals of $500,700,300,800,900$ hours, whence $$\mu = \frac{100}{500} + \frac{80}{700} + \cdots + \frac{400}{900} = 1.2545\ldots\,.$$

  2. The standard deviation is $\sqrt{1.2545\ldots} \approx 1.12.$ You can use this immediately (using only mental arithmetic) to bound the chances of large numbers of events. For instance, Cantelli's Inequality implies the chance of $5$ or more events, which is $(5-1.2545)/1.12 \approx 3.3$ standard deviations beyond the mean, cannot exceed $1/(1+3.3^2) \approx 1/12 \approx 8\%.$

  3. If, say, you wanted a $q=95\%$ symmetric interval in which the number of events is likely to fall, set $\alpha_l=\alpha_u=2.5\%$ and locate the $0.025$ and $0.975$ quantiles of the Poisson$(1.2545)$ distribution at $0$ and $4,$ respectively. The actual chance that the number of events will be between $0$ and $4$ inclusive is $99.1\%.$


To perform the calculations in the example I used the following commands in the statistical calculator R.

#
# Specify the problem.
#
intervals <- c(100, 80, 130, 50, 400)
averages <- c(500, 700, 300, 800, 900)
q <- 0.95
#
# Compute some properties of the distribution of the total number of events.
#
mu <- sum(intervals / averages)                    # Expected number
alpha.u <- alpha.l <- (1 - q)/2                    # Upper and lower tolerances
tl <- qpois(c(Lower=alpha.l, Upper=1-alpha.u), mu) # Limits
coverage <- diff(ppois(tl + c(-1,0), mu))          # Probability within limits 
names(coverage) <- "Coverage"
#
# Display the answers.
#
(tl) # The limits
(signif(c(Expected=mu, SD=sqrt(mu), `Nominal coverage`=q, coverage), 3))
| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I got lost in the last sentence. Expected value is 1.2545 with 95% symmetric interval <0;4>. How should I understand that actual chance is 99.1%? $\endgroup$ – Tomas Greif Oct 17 '19 at 14:48
  • 1
    $\begingroup$ Please see the calculation in the example: it computes the chance that a Poisson variable with mean $1.25$ lies between the lower and upper limit, inclusive, by computing the chance it is less than or equal to the upper limit (via ppois) and subtracting (via diff) the chance it is strictly less than the lower limit. $\endgroup$ – whuber Oct 17 '19 at 14:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.