I understand that random sampling is required for the purpose of creating an unbiased sample with the same characteristics as the population. I am confused about whether random sampling is required for IID data. Consider the following thought experiment. I toss a fair die a million times. Each time I toss the die, I note the number that appears on the die on a piece of paper and put the paper into a bowl. The numbers in the pieces of paper in the bowl are now realizations of IID random variables. The tosses are independent and each time the outcome is one of the numbers 1 to 6 with probability of 1/6. Now suppose I DO NOT shuffle the pieces of paper in the bowl and conveniently pick 10 pieces of paper from the top of the bowl. Isn't my sample an IID sample ? Is there any need to shuffle the pieces of paper in the bowl given the fact that I know that the random process that generated the population data is stationary and therefore any sample that I draw from the population is a realization of IID random variables ?
-
1$\begingroup$ You have essentially added an additional layer of randomness, and then said that it's not random. But the dice rolls were random, and then you shuffled them, and they are still random. If you remove the shuffling stage, they don't stop being random. $\endgroup$– Jeremy MilesCommented Dec 22, 2020 at 5:07
-
$\begingroup$ Thanks for your answer. What about my original question ? Is there any need to randomize the sampling process for data that has been generated by a stationary process ? $\endgroup$– TirthankarCommented Dec 22, 2020 at 5:12
1 Answer
Now imagine that you tossed your coin a million times, wrote down the numbers on pieces of paper, but before you took your sample, your grandmother decided to tidy up the pieces and sorted them. Nothing has changed about the numbers, but they are not shuffled anymore. Now if you picked “first ten” your sample would be obviously biased, since likely it wold contain all the same pieces.
We don’t sample because data is non-random (sampling wouldn’t help much in here), but because it’s not randomly scattered. For example, white people tend to live in more white areas, so taking “first ten” people at some area would be a racially biased sample. If you looked at “first ten” customers that visit your website at 5AM, those are likely not the same ones as those who visit it during the day, or in the evening. Same applies to many other kinds of data. Data often comes clustered, not scattered.
-
$\begingroup$ Thanks a lot @Tim for your answer. I have just one doubt. Technically speaking, the sample that is drawn from the sorted pieces of paper: is it still an IID sample or is it just a realization of identically distributed random variables that are dependent on each other due to the bias introduced by the sorting process ? $\endgroup$ Commented Dec 22, 2020 at 9:08
-
$\begingroup$ @Tirthankar if you took "first ten" from sorted heap, there would obviously be a dependence, since there would be a high chance than all, or at least most, samples would be exactly the same. Another example: say that you toss a coin but discard all non-heads, the sample would be deterministic. What you care about is the data you have, the fact that at some point there was something i.i.d. related to creating it is of less importance. That's why statisticians pay so much attention in defining what exactly is their population and sample, since it is easy to draw falce conclusions otherwise. $\endgroup$– TimCommented Dec 22, 2020 at 9:30
-
$\begingroup$ Thanks a lot @Tim for clarifying my doubt. $\endgroup$ Commented Dec 22, 2020 at 10:07