1
$\begingroup$

I am interested in fitting a Poisson/negative binomial distribution to estimate the number of times a phenomenon happens within a period, let's just say 10 years. I can count the events from monthly reports, but unfortunately, there are reports missing. So for one sample, I might have 120 observational slots, but for some others I might have 30. The event can happen if it is not observed.

The missing slots pattern is random (ie not correlated between samples), and it can result in anything from a nearly complete observational record to a very decimated one.

How can I cope with this?

$\endgroup$
3
  • 1
    $\begingroup$ If there is an underlying Poisson process or a Negative Binomial Lévy process, the distribution of the number of events within a period can be related to the effective duration which may be known, possibly with error. For instance, if you know that within a month the process was only observed during 20 days, this can be used in a GLM with appropriate link. $\endgroup$
    – Yves
    May 1, 2013 at 17:26
  • $\begingroup$ @Yves We assume that the maximum temporal resolution is a month. The day of month when the event happens is of no consequence here. $\endgroup$
    – Jose
    May 2, 2013 at 10:47
  • 1
    $\begingroup$ So why not simply omit the missing observations in the estimation e.g., using Maximum Likelihood? $\endgroup$
    – Yves
    May 2, 2013 at 11:54

3 Answers 3

2
$\begingroup$

Sounds like your problem is one where you have a fixed rate at which the event happens (after adjusting for covariates), but one unit you observe say 10 years while another for only 1 year. This is a fairly standard problem for this type of model, and the "offset" or an "exposure" is designed for that problem. Once you know those keyword, it is easy to find more about that in any textbook dealing with count data. As a textbook I like http://www.stata.com/bookstore/regression-models-categorical-dependent-variables

$\endgroup$
0
$\begingroup$

You need to think more about the data and provide more information so we can help:

  1. is the missing pattern consistent in different time slot?

From the information you provided, each time slot has random missing pattern, I assume you mean the missingness of a certain slot will not correlate with other time slots.

  1. what is the fraction of missing? if this fraction is small enough, you may consider throw them out or just fill in the mean value.

From your information, one way I can think of is to estimate Poisson/negative binomial parameters from completes observations. e.g. X_ij where i means sample and j means time points. You first select non-missing X_ij, then the mean of these non-missing X_ij can be thought as an estimator under the Poisson distribution.

Note: this naive method assumes random missingness and independence between time slots. For violations of these assumptions, you will need advanced techniques (i.e. imputation + time series).

$\endgroup$
2
  • $\begingroup$ Added some more detail, hope that helps $\endgroup$
    – Jose
    May 1, 2013 at 16:54
  • 2
    $\begingroup$ Welcome to the site, @zhanxw. At present this is actually a comment asking for more info, not an answer. Since the OP has provided more info, would you expand this into a proper answer? You could also make it into a real answer by explaining how this information is needed to address the situation. Note that, if this cannot be turned into an actual answer, it may need to be deleted. Since you're new here, you might want to read our FAQ, which discusses things like this. $\endgroup$ May 1, 2013 at 17:08
0
$\begingroup$

What it sound like you have is censored data, specifically right censored data. The events occurred, but they were not observed. Censoring (statistics)

If you are using R the fitdistrplus package that has exactly what you need. It has the function fitdistcens which can fit data with right, left and interval censoring.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.