In both the family-wise error rate (FWER) and false discovery rate (FDR) literature, particular methods of controlling FWER or FDR are said to be appropriate to dependent or independent tests. For example, in the 1979 paper "A Simple Sequentially Rejective Multiple Test Procedure", Holm wrote to contrast his step-up Šidák method versus his step-up Bonferroni control method:
The same computational simplicity is obtained when the test statistics are independent.
In "Controlling the False Discovery Rate" by Benjamini and Hochberg (1995), the authors write:
Theorem 1. For independent test statistics and for any configuration of false null hypotheses, the above procedure controls the FDR at $q^{*}$.
Later, in 2001, Benjamini and Yekutieli write:
1.3. The problem. When trying to use the FDR approach in practice, dependent test statistics are encountered more often than independent ones, the multiple endpoints example of the above being a case in point.
Which particular meanings of dependent an independent are these authors using? I would be happy for formal definitions of what makes tests dependent or independent from one another if they accompany a plain language explanation.
I can think of a few different possible meanings, but I don't quite grok which, if any, they might be:
"Dependent" means multivariate tests (i.e. many dependent variables with the same or similar predictors); independent means univariate tests (i.e. many independent variables, one dependent variable).
"Dependent" means tests based on paired/matched subjects (e.g. paired t test, repeated measures ANOVA, etc.); "independent" means a unpaired/independent samples study designs.
"Dependent" means that the probability that a test is rejected is correlated with the probability that another test is rejected, and "positive dependency" means that this correlation is positive; "independent" means rejection probabilities are uncorrelated.
References
Benjamini, Y. and Hochberg, Y. (1995). Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. Journal of the Royal Statistical Society. Series B (Methodological), 57(1):289–300.
Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Annals of Statistics, 29(4):1165–1188.
Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(65-70):1979.