I begin with a matrix of features (~30000 rows) vs cases (columns). I have binary - TRUE or FALSE - data in each cell indicating whether there is an event spanning the feature for that case. Next, I sum each row to get the total number of cases from the sample set that have an event for each feature. I will call this c for each feature. Next, I retrieve a list of features. How can I test whether c (the total number of cases with an event for a given feature) is significantly greater than c among the feature population in general? Furthermore, is there some way of calculating some uncertainty measure of my result based on the fact that only known features associated with a particular process are in the list.
NB. My attempt (probably wrong and overly complicated):
- Bootstrap with replacement 100000 times to get 100000 samples each of size 1023 from the 30000 features. The 1023 features (for example) are excluded from the population from which the features for each sample are drawn.
- Use
var.test
across c for each sample vs the 1023 feature list. If the test does not produce a significant value, that sample can be used in step 3. Samples that produce a significant value are discarded. This is done to satisfy the assumption of the Wilcoxon test that the variances between the two test samples are similar. - Do a Kolmogorov–Smirnov test (
ks.test
) for each sample brought forward vs the 1023 feature list. If the test does not produce a significant value, that sample can be used in step 4. Samples that produce a significant value are discarded. This is done to satisfy the assumption of the Wilcoxon test that the distributions between the two test samples are similar. - Do a Wilcoxon test (
wilcox.test
) for each sample brought forward vs the 1023 feature list. If the mean (or median?) p-value for these tests is significant, I can say that c is significantly greater for this set of features than it is for features that do not belong to this set.