I have the following problem: I have performed 3200 correlations between a certain measure of connectivity and 1 behavioural outcome in a sample of subjects. As the single correlations (represented by R and p) are not independent from each other, Bonferroni or FDR correction (at given p-values) failed to leave any significant correlations.
So I tried to follow this permuation approach: While permutating the available subject's connectivity data on the one hand, and also permutating the behavioural data on the other , I created a new pseudo-sample (in which subjects connectivity and behaviour are not matching anymore) for which I calculated all correlations again (3200). I did this 10000x and thus generated 10000x3200 random p-values based on the actual data set. Having ordered all p-values (low-to-high) I looked at the lowest 5% of those p-values (alpha): Random correlations would reach a p-value of 0.01 by 5% chance. In another group, the 5% lowest p-value would only reach 0.052.
My question would be whether this would be a plausible way to correct for multiple correlations in a dataset with lots of non-independent single correlations. Particularly, is it reasonable to use an adjusted p-value that is even higher than the original one, 0.05. Maybe the chance to reach 0.05 on that data set is even lower than 5% due to variance etc.
I am looking forward to your answers and comments.
Thank you very much