# How to Test Sampling Methods

Suppose I have a population, $$P$$, of 100 individuals and I am devising 3 algorithmic methods to sample 20 people from said population: $$A$$, $$B$$, and $$C$$. We know that $$C$$ is a random, unbiased algorithm.

Suppose the outputs for each algorithm are 100-length lists of 0's and 1's. Binary data where a 1 indicated that an individual is selected for a sample. There are 20 1's in each list:

$$A(P)$$ = $$[0, 1, 0, 1, ... , 1]$$

$$B(P)$$ = $$[1, 0, 1, 1, ... , 0]$$

$$C(P)$$ = $$[1, 0, 0, 1, ... , 1]$$

If I know that C is random, but I suspect that $$A$$ is not random and that $$B$$ is random, how can I go about testing this? I initially tried McNemar's test due to having paired nominal data. But then realized the statistic would always be 0 since my each list contains exactly 20 1's. Is Fisher's Exact test the best test for this type of experiment? How can I test a suspicion that $$B$$ is likely random (or similar to $$C$$) and show that the opposite is true for $$A$$?

• It's impossible to discern whether any sample is random based on this information. For instance, after obtaining sample $C,$ the experimenter might have numbered the population members so that the first 20 are in $C,$ whence $C(P) = (1,1,\ldots,1,0,\ldots,0).$ – whuber Jan 3 at 23:36

Assuming, that the sequence of items is not important and has not been rearranged after one of the selection processes (see the comment of whuber), this sounds like a classical application of the Wald-Wolfowitz runs test, see Runs Test for Detecting Nonrandomness or Wald-Wolfowitz runs test. For example, in your case with $$N_+=20$$ and $$N_-=80$$ one gets a mean number of runs $$\mu = 33$$ with a standard deviation $$\sigma = 3.17$$, which can be used as basis for the hypothesis test. But please check, that this is based on the randomness / unbiasedness you are assuming.