Suppose I have a population, $P$, of 100 individuals and I am devising 3 algorithmic methods to sample 20 people from said population: $A$, $B$, and $C$. We know that $C$ is a random, unbiased algorithm.

Suppose the outputs for each algorithm are 100-length lists of 0's and 1's. Binary data where a 1 indicated that an individual is selected for a sample. There are 20 1's in each list:

$A(P)$ = $[0, 1, 0, 1, ... , 1]$

$B(P)$ = $[1, 0, 1, 1, ... , 0]$

$C(P)$ = $[1, 0, 0, 1, ... , 1]$

If I know that C is random, but I suspect that $A$ is not random and that $B$ is random, how can I go about testing this? I initially tried McNemar's test due to having paired nominal data. But then realized the statistic would always be 0 since my each list contains exactly 20 1's. Is Fisher's Exact test the best test for this type of experiment? How can I test a suspicion that $B$ is likely random (or similar to $C$) and show that the opposite is true for $A$?

  • 1
    $\begingroup$ It's impossible to discern whether any sample is random based on this information. For instance, after obtaining sample $C,$ the experimenter might have numbered the population members so that the first 20 are in $C,$ whence $C(P) = (1,1,\ldots,1,0,\ldots,0).$ $\endgroup$ – whuber Jan 3 at 23:36

Assuming, that the sequence of items is not important and has not been rearranged after one of the selection processes (see the comment of whuber), this sounds like a classical application of the Wald-Wolfowitz runs test, see Runs Test for Detecting Nonrandomness or Wald-Wolfowitz runs test. For example, in your case with $N_+=20$ and $N_-=80$ one gets a mean number of runs $\mu = 33$ with a standard deviation $\sigma = 3.17$, which can be used as basis for the hypothesis test. But please check, that this is based on the randomness / unbiasedness you are assuming.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.