I have some data resulting from a simulation that consists of several groups, each containing a single real datapoint and a variable number of matched controls. I take the rank of each real value within its distribution of controls, and normalize the ranks to between 0 and 1.
Now I would like to test whether, across all groups, the ranks of my real datapoints are different to what I would expect by chance. However, I'm unsure about how to do this, since the distribution of possible normalized ranks across groups is discrete with uneven quantization (due to the variable number of control datapoints in each group).
I expect the CDF for the null distribution to look a bit like this:
I have considered the possibility of bootstrapping a null distribution by drawing randomly from each set of possible normalized ranks, then doing a 2-sample K-S test on the real and null distributions. Is this the way to go, or would there be a more appropriate test?
Edit: I've also posted a more complete description of the problem I'm trying to solve here.