I am collecting Twitter data through a source that allows me to take a statistical sample of tweets that match a filter. I want to determine the appropriate sample rate in order to achieve a certain statistical confidence in my estimate of the actual number of tweets that would match that filter.
For each tweet, the sampler calculates a new random floating point number uniformly distributed in the range [0,100], then compares that number to a given threshold (e.g. 1.6 for 1.6%). If the number is less than the threshold, the tweet is sampled; otherwise, it is discarded.
Let's say @justinbieber gets 500,150 mentions per day. I filter first to get all tweets that mention @justinbieber, then I sample at a probability of 1%. That should give me something like 5,001, which I'd then divide by 1% to get an estimate of 500,100 mentions/day.
There's two sources of error here:
1) Rounding error. I am necessarily reducing the precision of my count data, then extrapolating back to the original.
2) Sampling error. While the sampler uses uniformly distributed probability in the range, I may not get exactly the percentage of tweets I am requesting.
I want to calculate the appropriate sample probability I should use in order to achieve an estimate of twitter mention count with a certain confidence. For example, I might want to say with 95% confidence that my estimate is within 2% of the actual count. Given both sources of error here, what formulas or techniques allow me to calculate the proper probability to use in my sampler given an estimated per-day count?
The best solution should work equally well with small counts (<40) and large counts. It's certainly acceptable to sample at 100% for twitter handles with small counts, but there is a fixed cost per-tweet, and thus minimizing the number of tweets required is important when possible.