# Estimating repeat shoppers from an incomplete sampling

I'm trying to estimate how many people visited the farmers market once, twice, thrice, etc. in a given time period, using sampled data. We have interview data from approximately 50% of visitors as they entered the market which lets us identify them uniquely. For the purposes of this analysis, I'm assuming that the interviews randomly captured ~50% of visits; in reality the interviews were not random, but I want to start with a simpler problem first.

Within this dataset, I can identify how many people visited once, twice, thrice, etc. But, intuitively, I think the nature of sampling will lead me to underestimate the number of people who visited more than once. I've tested this intuition by randomly cutting the dataset in half a few times - starting from the full dataset (50% of visits), going down to 6.25%, and find that the larger datasets have more multiple-visit people in it (see below).

However, I am unsure what happens as I go from 50% of the visits to 100% of the visits. Can you help me come up with a statistical framework to do that projection?

PS - I feel that the Birthday Problem is informative here, but I can't think of how to apply it!

                     % of visits from people who visited at most.
% of visits sampled  once     twice    thrice
6.25%                83%      96%      99%
12.50%               82%      96%      99%
25%                  77%      94%      98%
50%                  67%      88%      95%
100%                 ?        ?        ?


I'm trying to think about how to apply the mark-recapture framework Gael mentioned. I agree that there is a similarity if I simplify it to the percentage of all visits from individuals who visited only once -- in essence, the population size in terms of this framework. What I'm struggling with is how to think about my sampling technique. I know how many total visits there (say, 25,000). I know that out of the 12,500 visits we "marked" in a continuous 50% sampling of visits, there were about 10,000 unique individuals, with 7,000 being marked once, and the remainder being "marked" 2, 3, or more times. I can't fit this into the simple two-stage mark-recapture model, and I'm thinking I need to do some kind of a Poisson regression (based on the wikipedia entry).

• is the data collected by asking "How many times have you visited the market?", or by recording their id and later tallying how often you recorded them? – Cam.Davidson.Pilon May 13 '13 at 17:50
• good question too! – Cam.Davidson.Pilon May 13 '13 at 17:51
• I think this bears some resemblance to what is know in ecology as “capture/recapture” or “mark and recapture”. Using these keywords might help you find relevant information. – Gala May 13 '13 at 19:00
• @Cam.Davidson.Pilon It is done by recording their ID and later tallying how often I recorded them. Interestingly, from another source, I have a "how many times..." type datapoint, but considered that even harder to work with as I only know the number up to that point in time that it was measured, but don't know what that number looks like later in time. – Jonathan May 13 '13 at 21:01
• I just saw the question stats.stackexchange.com/questions/2466/… so I will be playing with that to see if I can apply it here... – Jonathan May 15 '13 at 0:45