Imagine we have a webserver, which serves a total of N static URLS.
There are users visiting the URLs every day. At the end of each day, we have data like this:
URL1 --> 500 visits URL2 --> 450 visits URL3 --> 370 visits URL4 --> 330 visits ... URLN-1 --> 1 visit URLN --> 0 visits
Then, we take a weighted sample out of these URLs. Each URL is included in the sample with probability proportional to the number of visits.
URLN will never be sampled, And
URLN-1 (which was only visited once that day) is
500 times less likely to be selected than
Why we sample?
Each URL leads to a page which either has a
white background or a
black one. Assume that determining this background is a time-consuming task which we can't automate it. Hence, we would like to sample a small number of URLs, manually check their colors, and produce such an estimate for the whole set of URLs:
- How likely will a URL visit lead to a page with a white background?
This is easy: given a set of sampled URLs, for which we have manually determined the colors, we compute the following:
(SUM of visits of white URLs in the sample) / (SUM of all visits in the sample)
E.g. if this produced a number like 0.40, we could say "we estimate that 40% of URL visits lead to a page with white background".
But here's the tricky part:
- How can we compute confidence intervals (say at 95%) for this quantity?
I've been reading various "resampling" or "bootstrap" techniques (wikipedia), for example Jackknife or Bootstrap, or even Poisson Bootstrap which seems a good choice for large data that need to be processed in a distributed way.
To get a 95% confidence interval with such a method, all I'd have to do would be to compute the 2.5% and 97.5% quantiles of the generated estimates.
But all these methods appear to assume that the sampled observations (URLs here) are selected with the same probability. In my case, they aren't, they are selected with probability proportional to the number of visits.
And again, let's assume that I can't avoid the use of weights, I can't process my data to remove the weights, e.g. by replacing "URL111 --> 2 visits, URL999 --> 3 visits" with [URL111, URL111, URL999, URL999, URL999]. Sure, in that case, these bootstrapping techniques would work without modification.
So: Is there a way to augment one of these techniques to take account of sampling weights, and to produce confidence intervals?