# Estimating repeat shoppers from an incomplete sampling

I'm trying to estimate how many people visited the farmers market once, twice, thrice, etc. in a given time period, using sampled data. We have interview data from approximately 50% of visitors as they entered the market which lets us identify them uniquely. For the purposes of this analysis, I'm assuming that the interviews randomly captured ~50% of visits; in reality the interviews were not random, but I want to start with a simpler problem first.

Within this dataset, I can identify how many people visited once, twice, thrice, etc. But, intuitively, I think the nature of sampling will lead me to underestimate the number of people who visited more than once. I've tested this intuition by randomly cutting the dataset in half a few times - starting from the full dataset (50% of visits), going down to 6.25%, and find that the larger datasets have more multiple-visit people in it (see below).

However, I am unsure what happens as I go from 50% of the visits to 100% of the visits. Can you help me come up with a statistical framework to do that projection?

PS - I feel that the Birthday Problem is informative here, but I can't think of how to apply it!

                     % of visits from people who visited at most.
% of visits sampled  once     twice    thrice
6.25%                83%      96%      99%
12.50%               82%      96%      99%
25%                  77%      94%      98%
50%                  67%      88%      95%
100%                 ?        ?        ?


I'm trying to think about how to apply the mark-recapture framework Gael mentioned. I agree that there is a similarity if I simplify it to the percentage of all visits from individuals who visited only once -- in essence, the population size in terms of this framework. What I'm struggling with is how to think about my sampling technique. I know how many total visits there (say, 25,000). I know that out of the 12,500 visits we "marked" in a continuous 50% sampling of visits, there were about 10,000 unique individuals, with 7,000 being marked once, and the remainder being "marked" 2, 3, or more times. I can't fit this into the simple two-stage mark-recapture model, and I'm thinking I need to do some kind of a Poisson regression (based on the wikipedia entry).

• is the data collected by asking "How many times have you visited the market?", or by recording their id and later tallying how often you recorded them? Commented May 13, 2013 at 17:50
• good question too! Commented May 13, 2013 at 17:51
• I think this bears some resemblance to what is know in ecology as “capture/recapture” or “mark and recapture”. Using these keywords might help you find relevant information.
– Gala
Commented May 13, 2013 at 19:00
• @Cam.Davidson.Pilon It is done by recording their ID and later tallying how often I recorded them. Interestingly, from another source, I have a "how many times..." type datapoint, but considered that even harder to work with as I only know the number up to that point in time that it was measured, but don't know what that number looks like later in time. Commented May 13, 2013 at 21:01
• I just saw the question stats.stackexchange.com/questions/2466/… so I will be playing with that to see if I can apply it here... Commented May 15, 2013 at 0:45