I read these posts and their references:
Recommended reading for understanding when the bootstrap will fail?
Multiple samples vs. bootstrapping
but I am failing to understand. Consider the following example:
You want to know how many people speak each language on earth as a fraction of the total population. To determine this statistic, you take a sample of people, 1 million. Then you resample this population a thousand times creating 1000 sub-samples of 1000 people each. (or does the reampling have to a million samples of a million each? or does it make a difference? But this is an aside not germane to my fundamental question)
Then you measure the language distribution in each sub-sample. Then you collect the data for a particular language, say English. And you 'do statistics' on this dataset - for example, to calculate the average fraction of people who speak English - you sum the fraction of English speakers from all the sub-samples and divide by the number of sub-samples. My understanding is that bootstrapping says this average should reflect the average fraction on the planet. (Please do not be offended by my naive question and its even more naive exposition.)
Now, what if you collected your initial sample of 1 million from the web pages on the internet? Then your initial sample would not be representative at all of the planet-since the internet is mostly in English. Obviously, this is a trivial example and the method of obtaining the initial sample is biased in an obvious way. In 'real working problems', the method of sampling may have biases which are not so obvious. The way the initial sample is collected has tremendous bearing.
Is there a way to know when the initial sample is not representative of the population? For example in the simple problem above - would you expect that the distribution of averages is a normal distribution but you found the distribution of average fractions is skewed or bumpy - then would you suspect anomaly in the original sample?
