Can I combine random samples and still get a random sample?

Suppose I have a set $$X$$ with partition $$X_1, \ldots, X_k$$. Suppose $$Z_i = (z_{i1}, \ldots, z_{iN/k})$$ is a random sample of $$N/k$$ elements from $$X_i$$ and $$Z$$ is a random sample of $$N$$ elements from $$X$$.

Can I treat "union" $$\cup_i Z_i := (z_{11}, \ldots, z_{1N/k}, \ldots, z_{k1}, \ldots, z_{kN/k})$$ of random samples from $$X_i$$ as a random sample of $$k \cdot N/k = N$$ elements from $$X = \cup_i X_i$$?

• You normally cannot operate with "unions" on truly random samples, because they are not sets, i.e., they can contain repeating elements of the initial set. But if you just joint those separate independent samples together, they will constitute a random sample. But importantly it will be random from the perspective of the time before you take the first sample. If you already have a couple of small samples and decide to continue sampling, the total large sample won't be truly random, because it will contain a part, which had been already certain.
– Alex
Commented Oct 13, 2022 at 19:04

The process you describe is not equivalent to random sampling of N elements from $$X$$. Suppose $$N$$=100 and $$k$$=5, which means each $$Z_i$$ has 20 elements. Suppose further each $$X_i$$ has 40 elements. If the random sample chose elements 1 through 19 from $$X_1$$ then it can only choose one of the rest of the elements (elements 20 through 40). Which means that your process introduces dependencies which aren't there in the straightforward sampling procedure.

The dependencies can be reduced (but not completely eliminated) by making sure the ratio $$\frac{N/k}{|X_i|}$$ is small for every $$i$$.

• Thank you! And if my ratio (N/k)/(|X_i|) is about 0.000001, will it be okay? Commented Oct 13, 2022 at 20:27
• Well, strictly speaking, that depends on the use case, but for that ratio I think it safe to assume that you're good. Commented Oct 13, 2022 at 20:34