This is an obvious question - random sampling seems intuitive as it makes each element having an equal chance of being selected, intuitively reducing possible biases. If someone was to calculate the "lift" of a model, they would intuitively compare it against random selection. Are there any theorems which explicitly state the importance of random sampling in being able to generalize the results (more so than if elements were sampled systematically - e.g. if elements are sampled strictly every 30 minutes, this may cause a bias)?
2
-
1$\begingroup$ Sampling something each 30 minutes in many cases could be equivalent to random sampling. Similar example: you ask every 10th passing pedestrian to fill in your questionnaire - in many cases there is no reason to consider such sample to be non-random sample of pedestrians. Pseudo random number generators are also deterministic functions producing "random" outcomes. $\endgroup$– TimCommented Nov 5, 2015 at 15:54
-
1$\begingroup$ @tim: On the other hand, if you do the sampling 48 times less frequently, it probably wouldn't be representative. Unless you know that there's no periodicity at the sample frequency, or at a multiple of the sample frequency (and you can't know that for sure), then you can't assume it's random. $\endgroup$– naught101Commented Apr 18, 2016 at 6:22
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
There are many papers and book that study this in different settings. One example, would be with Markov Chain Monte Carlo (MCMC) where the samples generated from the posterior distributions are auto-correlated. Time series can be similar. In this case you can measure "effective sample size" which basically measures how much information you have lost by not having proper full random sampling. See for example this question.
There won't be a really strong general result for all settings, as there are so many ways to deviate from fully random sampling.
Note that the opposite is also often true, for example with imbalanced groups stratified sampling can be much more effective then purely random sampling. If you study the effect of age on a dependent variable you might want so sample people so you have a wide range of age.
$\begingroup$
$\endgroup$
It sounds like you want a specific answer to a general question. I'd say this is covered by the law of large numbers, but if you want a more specific study, I'd look at Gaussian convolution operations. It is easy to demonstrate that the average value of the sum of a number of similar Gaussian curves trends quickly to a delta function with a vanishing standard deviation.
For theorems that "state the importance of random sampling", I would look more at various hypothesis testing procedures to quantify that uncertainty.
