I'm comparing $m$ p-values, where $m$ is very large. Every p-value comes from a two-sample $t$-test of two groups of size $n_1$ and $n_2$, where the $n_i$ are very small ($2$ or $3$). This leads to a $(m\times(n_1+n_2))$ data matrix $X$.
Thus I would like to employ a permutation test to assess the significance of my p-values. I've got two questions regarding this problem:
If I permute the sample labels for every test (e.g. with $n_1=n_2=2$ this gives 24 different label assignments) and calculate the corresponding $t$-statistic, I can evaluate if the original label assignment leads to a significant result by counting the number of permutations that lead to a larger $t$-statistic. Do I still need to correct for multiple testing?
Since the $n_i$ are very small, does it make sense to permute the test labels instead? I.e., I draw $B$ times a set of $n_1+n_2$ values, where I draw one value from each column of $X$. This gives a null distribution based on these $B$ sets and their corresponding $t$-statistics. If I now compare the original $t$-statistics to this distribution, do I need to correct for multiple testing?