I have just read this paper about different test statistics one could use in Information Retrieval (IR), discussing whether Wilcoxon, Sign-test or Student t-test would be more reliable for comparing two IR systems.
According to the paper I read, the Sign and Wilcoxon tests do not take into consideration the statistic of interest (which is the difference in average precision); and by approximating them (both in different ways), we get a higher difference in p-value (generally higher), this leads to false positives and negatives.
In addition, the authors discourage using the Wilcoxon and Sign tests since they were built when computing randomized tests or Student’s t test was very expensive.
However, at the end of the paper, they mention that a small sample size (smaller than 50), "violations of normality may result in errors in the t-test."
So my questions come here: When we use Wilcoxon, Sign test, and T-test, which should we rely on? Are the points that I am trying to make valid?