It is common practice in neural decoding studies to compare the decoding accuracy on a data set with the accuracy on the label-shuffled data set, and then use a statistical test to determine whether decoding performance on the unshuffled data is significantly higher.
Suppose a dataset consists of $n$ accuracy scores on $n$ sessions or trials, and $n$ accuracies on the corresponding shuffled data. My question is what would be the most appropriate statistical test to address the difference between the two population means.
My confusion arises mainly because some studies use a Wilcoxon rank sum test (Mann-Whitney U-test), for example this paper, which is a test for independent samples. Others, see for example this article, use a paired t-test, which as the name suggests is tailored to paired observations.
I would say that a score computed on unshuffled and shuffled data is a dependent observation, as there is a one-to-one correspondence between points in each pair, and thus a Wilcoxon rank sum would be inappropriate (the Wilcoxon signed rank test would be a paired-samples nonparametric alternative). I have also noticed that the Wilcoxon rank sum produces lower p-value than the paired t test.