I'm trying to estimate how many people visited the farmers market once, twice, thrice, etc. in a given time period, using sampled data. We have interview data from approximately 50% of visitors as they entered the market which lets us identify them uniquely. For the purposes of this analysis, I'm assuming that the interviews randomly captured ~50% of visits; in reality the interviews were not random, but I want to start with a simpler problem first.
Within this dataset, I can identify how many people visited once, twice, thrice, etc. But, intuitively, I think the nature of sampling will lead me to underestimate the number of people who visited more than once. I've tested this intuition by randomly cutting the dataset in half a few times - starting from the full dataset (50% of visits), going down to 6.25%, and find that the larger datasets have more multiple-visit people in it (see below).
However, I am unsure what happens as I go from 50% of the visits to 100% of the visits. Can you help me come up with a statistical framework to do that projection?
PS - I feel that the Birthday Problem is informative here, but I can't think of how to apply it!
% of visits from people who visited at most. % of visits sampled once twice thrice 6.25% 83% 96% 99% 12.50% 82% 96% 99% 25% 77% 94% 98% 50% 67% 88% 95% 100% ? ? ?
I'm trying to think about how to apply the mark-recapture framework Gael mentioned. I agree that there is a similarity if I simplify it to the percentage of all visits from individuals who visited only once -- in essence, the population size in terms of this framework. What I'm struggling with is how to think about my sampling technique. I know how many total visits there (say, 25,000). I know that out of the 12,500 visits we "marked" in a continuous 50% sampling of visits, there were about 10,000 unique individuals, with 7,000 being marked once, and the remainder being "marked" 2, 3, or more times. I can't fit this into the simple two-stage mark-recapture model, and I'm thinking I need to do some kind of a Poisson regression (based on the wikipedia entry).