4
$\begingroup$

The Poisson distribution arises when events are counted within a specified interval.

I've recorded the number of events each month (I'll not discuss what these events represent). This appears to meet the assumptions of the Poisson distribution.

However, within each month, I have done a variable number of hours. And more hours tends to lead to more events. Am I breaking the assumptions of the Poisson distribution?

$\endgroup$
  • 2
    $\begingroup$ The Poisson distribution can be thought of as modeling a process both having a fixed average rate of events per unit time and having independence between events. So as you pose the question, it seems that a Poisson process would have more events if you examined longer intervals of time. Did you mean something else by your question? Maybe you could show a plot of some troubling data. $\endgroup$ – EdM Jun 22 '15 at 17:57
  • 1
    $\begingroup$ I'm a bit confused by what you mean by "within each month, I have done a variable number of hours." $\endgroup$ – Ryan Simmons Jun 22 '15 at 18:09
  • $\begingroup$ Repeating my comment from below here, as suggested; Here's where i think the confusion is. What does "doing hours" mean? Please explain carefully, then i believe your question can be answered $\endgroup$ – Mark L. Stone Jun 24 '15 at 7:22
  • $\begingroup$ Imagine that each month you do a variable numbers of hours at work. In January you did 140 hours. However in February you did 165 hours (because you did some overtime). $\endgroup$ – luciano Jun 24 '15 at 11:31
  • $\begingroup$ @luciano Could you comment on my answer and explain why why using a count of hours exposure does not solve your problem? $\endgroup$ – Matthew Drury Jun 25 '15 at 19:13
7
$\begingroup$

You can deal with the unequal number of days in a month by using the concept of an exposure. A good reference for this is Data Analysis Using Regression and Multilevel Models chapter 6.2.

Here's an overview. Let's attempt to estimate, instead of the number of events per month, the number of events per day. This is better, because a day is always a fixed interval of time, while a month contains a variable amount of time. We can record the amount of time in each month by simply writing down the number of days it contains. If we would like instead to know the rate of events per hour, we only need to divide our estimated number of events per day by $24$.

So lets say we have data like

| Count of events in month | Number of days in month
+--------------------------+------------------------
| 0                        | 29
| 2                        | 31
| 0                        | 31
| 1                        | 30
| ...                      | ...

We can model this data by introducing $\lambda$, the rate of events per day, $y$ the count of events in the month, and $d$ the number of days in the month (the exposure). Then

$$ y \sim Poisson(d \lambda) $$

This checks out conceptually, if a month is twice as long, we expect twice as many events.

To fit $\lambda$ from such data, we can use a poisson glm (logarithmic link) with an explicit offset term to absorb the exposure. In R

glm(count_of_events_in_month ~ 1, offset = log(number_of_days_in_month), family=poisson)

This glm will result in a formula like

$$ log(E(y)) = \beta_0 + log(d)$$

which, after exponentiating becomes

$$ E(y) = e^{log(d)}e^{\beta_0} = d e^{\beta_0} $$

So $\lambda = e^{\beta_0}$ is our estimate for rate per day. As mentioned, $\frac{\lambda}{24}$ gives the estimated number of events per hour.

note: If instead of number_of_days_in_month, we included number_of_hours_in_month, we would have instead directly estimated the rate of events per hour.

$\endgroup$
  • $\begingroup$ +1 Note that since the OP mentions "number of hours" in the question, where "more hours leads to more events", those would be a useful measure of exposure for his specific problem. $\endgroup$ – Glen_b -Reinstate Monica Jun 23 '15 at 2:28
  • 1
    $\begingroup$ @Glen_b Added some comments on per-hour estimation. $\endgroup$ – Matthew Drury Jun 23 '15 at 3:55
  • $\begingroup$ @MatthewDrury Thanks accepted this as answer. One further question. Could I convert the number of events to a per hour rate and use month as a predictor in the model. R code to run this model would be something like glm(events/hour ~ month, family = 'poisson'). Note the lack of an offset term in this model $\endgroup$ – luciano Jun 26 '15 at 6:46
1
$\begingroup$

(As mentioned in the comments) the most basic assumptions for a Poisson distribution are that

  1. Events are independent.
  2. The rate of events per unit time is fixed.
  3. The variance of events per unit time is equal to the average events per unit time.

Or, with mathematics: if $X \sim Po(\lambda)$ is the variable we think we are observing with our sample of $(x_1, x_2, \ldots, x_n)$, then:

  1. $x_i$ represent the number of events counted in some fixed time, $t$
  2. The value observed in $x_i$ has no influence on that observed in $x_j$
  3. The $x_i$ and $x_j$ both came from $Po(\lambda)$ -- this is unfortunately circular
  4. $\hat{\mu} \approx \hat{\sigma}^2$ That is, $$ \frac{1}{n} \sum_{i=1}^{n} x_i \approx \frac{1}{n} \sum_{i=1}^{n} (x_i - \hat{\mu})^2 $$

Disregarding the last sentence -- which I cannot admit to understand -- you can rest assured that if your data satisfies points 0, 1, 2, & 4, it's probably safe to model it with a Poisson distribution.

Edit:

As mentioned in this answer's comments, a strict adherence to point #1 would necessitate that you "redefine" a month to be some fixed number of days (e.g. a "month" is 30 calendar days)...

Edit2:

It should also be stated that while point #3 is unhelpful, I can clarify it in the context of your data. #3 merely says that you believe that the process that made your data, isn't affected by the calendar month. So, if you data were counts of purchased cars, per month, #3 says that the counts you observe in, say, June and February, came from the same underlying distribution. In particular, that there are no seasonal affects.

For most consumer/biological data, seasonal affects are highly non-trivial, and make month-level Poisson models fairly naïve. An alternative way to go could be to model event occurrence on a day-by-day basis, but at a "seasonal level". E.g. estimate 12 Poisson parameters, $\lambda$, for each month of data...

$\endgroup$
  • $\begingroup$ 1. needs to be some fixed amount of time t. I think the OP's point is that each data point is the count in a given month, but since months have a varying number of days, this assumption is violated. $\endgroup$ – Matthew Drury Jun 22 '15 at 21:56
  • $\begingroup$ Oh, I see! Depending on how rare the occurrence are, an extra day may not actually make too much of a difference, but I understand... $\endgroup$ – StevieP Jun 22 '15 at 22:05
  • $\begingroup$ No I mean that in January there 10 events and I did 15 hours, in February there were 8 events and I did 10 hours and so on $\endgroup$ – luciano Jun 23 '15 at 5:39
  • 1
    $\begingroup$ Here's where i think the confusion is. What does "doing hours" mean? Please explain carefully, then i believe your question can be answered. $\endgroup$ – Mark L. Stone Jun 23 '15 at 22:47
  • 1
    $\begingroup$ You may want to post that comment in the question's comment section. Sadly, I'm just as lost as you. $\endgroup$ – StevieP Jun 23 '15 at 23:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.