Let me see if I can explain this problem in an intuitive way.
Suppose I have a lighthouse in the distance, and it's either flashing every 1 second, or every 5 seconds. I want to determine which it is.
I have a group of people that are going to independently (i.e., not at the same time) observe the lighthouse for 60 seconds each. They will write down times in seconds relative to the beginning of their observation when they see the light flash. (So, if they see the light at 8 seconds and 21 seconds, they'd write down 8 and 21.)
If my observers were perfect, this would be easy. But unfortunately, they are not. They're highly unreliable and most of them have a less than 10% chance of seeing any given flash. (Consider consecutive flashes independent. Seeing flash N does not affect the observer's chance of seeing flash N + 1.) And each observer has a different miss rate. Some might see the light 10% of the time, others might see it 1% of the time, etc.
The observers are also not perfect about measuring time. Sometimes, they'll be off by a second or two, though 95% of their measurements are off by 1 second or less. This property is the same for all observers - their time measurement abilities can be considered equal for the purposes of this problem.
Given this data, how can I design a statistical test to determine whether the lighthouse is flashing every 1 second or every 5 seconds?
My first intuition was to measure the gaps between sightings for each observer. So, if an observer saw the lighthouse at 6, 11, 26, and 51 seconds, that would produce gaps of 5, 15, and 25 seconds. I could then take these gaps, divide them by 5, and get the remainder. In this case, the remainders would be 0, 0, 0. I could then plug this into a chi-square test to see if the gaps-mod-5 are evenly distributed, or if they're concentrated around 0.
But there's two problems with this. First, you can't assume that the gaps-mod-5 should be evenly distributed between 0 and 4 in the lighthouse-flashing-every-second case, because long gaps should be relatively less likely than short gaps (since a long gap implies missing more flashes than a short gap). And the expected distribution of the gaps is dependent on the miss rate, which is different for each observer... So, when you assume the gaps-mod-5 should be evenly distributed between 0 and 4, the chi-square ends up rejecting a lot because you're seeing 0 less than 20% of the time, rather than more. But that's expected.
The second problem with using the remainders is that you lose some of your power. A gap of 45 seconds is good evidence for the lighthouse flashing every 5 seconds not just because it's a multiple of 5, but also because it implies that you would have missed 44 consecutive flashes in the 1-second case. If the observer in question sees the light 10% of the time, missing 44 consecutive flashes is highly unlikely. Of course, the miss rate is also unique to each observer, and it's an unknown value.
The other intuition I had was that you could estimate the miss rate for each observer for the 1-second case by taking the number of sightings and dividing by 60. If observer X saw the lighthouse twice, that's an implied miss rate of 58 / 60 = 97% if the lighthouse is flashing every second. You could then calculate for that particular observer an expected distribution of observed gaps, where frequency of each gap length should just be a function of the estimated miss rate. Then you could sum your expected and observed values across observers. Then you could run your chi-square. Unfortunately, I don't know how using estimates of your expected values would affect your chi-square results...
Does anybody have suggestions for another test that could be used to elegantly handle this problem? It has relations to the poisson distribution as well, but I'm not sure how to design a neat test to fit there either.