2
$\begingroup$

I read these posts and their references:

Recommended reading for understanding when the bootstrap will fail?

Multiple samples vs. bootstrapping

but I am failing to understand. Consider the following example:

You want to know how many people speak each language on earth as a fraction of the total population. To determine this statistic, you take a sample of people, 1 million. Then you resample this population a thousand times creating 1000 sub-samples of 1000 people each. (or does the reampling have to a million samples of a million each? or does it make a difference? But this is an aside not germane to my fundamental question)

Then you measure the language distribution in each sub-sample. Then you collect the data for a particular language, say English. And you 'do statistics' on this dataset - for example, to calculate the average fraction of people who speak English - you sum the fraction of English speakers from all the sub-samples and divide by the number of sub-samples. My understanding is that bootstrapping says this average should reflect the average fraction on the planet. (Please do not be offended by my naive question and its even more naive exposition.)

Now, what if you collected your initial sample of 1 million from the web pages on the internet? Then your initial sample would not be representative at all of the planet-since the internet is mostly in English. Obviously, this is a trivial example and the method of obtaining the initial sample is biased in an obvious way. In 'real working problems', the method of sampling may have biases which are not so obvious. The way the initial sample is collected has tremendous bearing.

Is there a way to know when the initial sample is not representative of the population? For example in the simple problem above - would you expect that the distribution of averages is a normal distribution but you found the distribution of average fractions is skewed or bumpy - then would you suspect anomaly in the original sample?

$\endgroup$
-1
$\begingroup$

Is there a way to know when the initial sample is not representative of the population?

The bootstrap cannot detect bias. Bias is a sampling problem which would have to be detected by some other tests.

For example in the simple problem above - would you expect that the distribution of averages is a normal distribution but you found the distribution of average fractions is skewed or bumpy - then would you suspect anomaly in the original sample?

Common sense suggests you look for indications of a problem.

$\endgroup$
  • $\begingroup$ Well, one of the uses Efron shows originally of the bootstrap is to bias correction ... $\endgroup$ – kjetil b halvorsen Dec 25 '15 at 23:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.