Hypothesis-testing on low frequency data

Question

I'm trying to figure out a smart way to test some data, and afterwards correct my assumption. So I'm looking for suggestions to which path to take.

I'm looking at how often birds crash into windows, and I have a complex model with many covariates describing the crash frequency of these birds. I have data on 10,000 birds over 10 years (2002-2011). All the birds crashing into these windows survive, and I'm able to follow each bird through time (post 2011). As the bird grows older the crash frequency decreases, I assume because the birds grow more experienced. Now, I know that I'm not able to account for all the variance in my data, so I wish to construct a rating system which I apply to each bird, thus adjusting my initially calculated crash frequency after each birds individual crash history (after 2011, when my model is made).

My problem is that the birds crash very seldom, with a year as a unit, crash frequencies vary between 0.15 - 0.25. And they may even crash multiple times a year.

Crash frequency from my complex model, at year 2012 is in parenthesis. Crash frequency goes down 0.02 each year, because the bird becomes one year older. All other covariates are the same, city, country, buildings etc.

Would there be a smart test, or some Bayesian approach, or some other technique to solve my problem? Kalman filter?

EDIT: Example: In 2012 I make my model on 10,000 birds, 10 years of data. I get a priori estimates for all the birds, from Poisson regression. Note that I'm talking about a priori in not the regular sense, because I have 10 years of data, but I wish to update the crash frequency estimate each year, to get a "a posteriori" estimate. More explicitly: I enter 2012, and I see results for each bird. This would maybe be too early to say anything certain regarding my a priori estimates - if they were too high or too low. But then 2013 comes, and I see new results. If some bird has crashed two years in a row, I might want to update his estimate. I would like some algorithm, model, or test, to check this each year...

EDIT 2: Is this a credibility theory problem, maybe?

Answer 1

So for each bird, the model is a Poisson process, where the lambda parameter is different for different birds?

In that case I suggest that you assume a suitable prior distribution for lambda, and then make a Bayesian fit to the observations to deduce a posterior distribution of lambda for each bird. The likelihood of the observations given these posterior distributions is then the quality indicator for the model.

For choosing a prior, first use a naive estimate of lambda for each bird. Then look at the empirical distribution of these. Then identify a distribution that fits it. If a gamma distribution is a reasonable fit then all the better because this is the conjugate prior for Poisson.

Only a Bayesian approach is really justifiable with sparse data. OK, assuming a prior distribution is a big assumption, but at least it is explicit. Any fit to sparse data involves prior knowledge.

Answer 2

For rare events, if you have a lot more data, you might try Poisson regression, for which the bird-time (person-time) of follow-up is known for each bird, along with the failure event (0,1) of crashing. Calendar year could be a variable as well as other experimental (categorical factors). Otherwise, like you say, contingency table testing would probably work, and be certain to use the Fisher exact test (p-value).