I have a set of data that is essentially bowls with marbles in them. Each bowl can contain anywhere from 3 to hundreds of thousands of marbles. Many more bowls contain small numbers of marbles than large numbers of marbles--if you arrange the bowls from fewest marbles to most marbles, when you reach the 99th percentile of bowl size you're in the 75th percentile of marbles--1% of the bowls contain about 25% of the marbles.

The population is ultimately bowls--we cannot look at a single marble, only a whole bowl of marbles. But we want our sample to represent both bowls and marbles well. Ideally we'd like a sample that represents both marbles and bowls well (although I'm not quite sure what "well" means here). Stated another way, we would like a sample that reflects the full range of bowl sizes.

The problem is that these needs are conflicting. If we sample bowls randomly, we end up sampling very few marbles since the vast majority of bowls contain very few marbles. If we randomly select marbles and make a sample by collecting the bowls containing all the marbles we selected (or we weigh each bowl's sampling probability by the number of marbles), we end up with only very large bowls and a sample that doesn't reflect the range of bowl sizes.

The superficial solution seems to be to take two samples and combine them--one random sample of the population of bowls, and one random sample of the population of marbles (where we take the entire bowl for each sampled marble).

But I can't help but think there's a better way to do this, with some kind of parameter that reflects the relative importance of a random sample of bowls vs. a random sample of marbles. Or at least some way to analyze the tradeoff based on the distribution of marbles across bowls.

And even with the superficial sample, I'm not quite clear how to balance the sample sizes to represent bowls and marbles "evenly". I think there is a better way to do this than simply stratifying by bowl size (which is the best idea I have right now) since there's likely a way to parametize the bowl/marble distribution.

I'm sure there's a better way to describe this problem, where you have a population where the members have a count value, where the counts are not evenly distributed across the population, and you want a good sample of both random population members and of population members reflecting the diversity of count values. Any clarity on a better way to describe this would be helpful, even if you don't have an answer, so at least I could more deliberately search for a solution.

Thank you!

  • $\begingroup$ It looks to the quite common problem of sampling families or sampling children. Both samples are interesting but they are from different populations, although they are formed by the same people. You need to decide if bowls or marbles are your population. $\endgroup$ – Pere Oct 17 '17 at 23:13
  • $\begingroup$ Not sure if this makes any sense so pls correct me if I sound absurd. If I understood right, by drawing "evenly", you meant that you want to draw in a jointly uniform fashion, is this right? If that's the case, perhaps you could use the probability integral transform? $\endgroup$ – user3119750 Oct 18 '17 at 19:13

I was able to seek the advice of a statistician who specializes specifically in sampling. The recommended approach was to stratify extremely aggressively--order the balls from fewest marbles to most marbles and then take a single example from each strata. This means you have a number of strata equal to the size of the sample.

This preserves the distribution of the data, but ensures that the long tail also gets sampled.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.