# Does random sampling from a dataset produce the same distribution as the original space?

Let's say we have a dataset $$D$$ with $$N$$ rows and $$M$$ columns. Each column is a feature. And for each feature $$X_1, X_2,..., X_N$$~ iid $$F_p$$ where $$F_p$$ is the distribution for feature p. Now let's do a random sampling which samples $$K$$ rows from $$D$$ and let's denote this sample as $$D^K$$.

So my question is: in $$D^K$$, for each feature p, does it follow the same distribution of this feature p in $$D$$? My instinct is that when $$M$$ is a large number, if $$K$$ is not large enough, then it cannot guarantee every feature p in $$D^K$$ has the same distribution as that in $$D$$. But I don't know if it is right. If it is right, how can I determine the minimum $$K$$ that's required?

Thanks!

• If you sample with replacement from the sample you're sampling from $\hat F$ rather than F. If you're randomly sampling the population the sample cdf converges to the population cdf, but resampling the sample cdf will converge to that, not to F – Glen_b Jul 12 at 3:51
• yea, I intended to say sample without replacement. – lun yu Jul 29 at 17:39
• If you sample without replacement it's even worse, you just get $\hat{F}$ every time. – Glen_b Jul 29 at 22:32

Explicitly, your $$D^k$$ sample will not follow the original population unless the population is discrete and $$D$$ perfectly recreates the discrete PMF. But with large enough sample sizes, you should get close.