is empirical FDR denominated by measurements or by experiments? I have a case-control experiment that produces one p-value of 0.002 for a given measurement (out of 100 measurements). If I shuffle the case-control labels and run the analysis 1000 times as a permutation test, 80 (or 8%) of my experiments will produce a p-value equal or lower to 0.002. In those 1000 pseudo-experiments, there were 100,000 measurements, but only 120 of them (or 0.12%) of them achieve this p-value.
Which is the real False Discovery Rate (8% or 0.12%)? What is the other rate called?
 A: The first value of 8% is just a p-value, relating to the combined null hypothesis that none of the 100 measurements are different between case and control.
You can reach this p-value by the following reasoning. Consider the null hypothesis that none of the measurement are different vs the alternative that at least one is different. Let's take the test statistic to be the minimum p-value obtained for any of the 100 measurements. You permute the labels, evaluate the test statistic for each permutation and the permutation p-value is 80/1000 = 0.08. (Actually, to be more precise, the permutation p-value is p = (80+1) / (1000+1), see Phipson & Smyth, 2010.)
The second value of 0.12% could be an estimated FDR under certain assumptions. To make this a valid estimator, you would need to be able to assume that the permutation distribution of p-values is the same for all of the 100 measurements that are not truly different between cases and controls. From the numbers you give, I suspect that this might not be a reasonable assumption.
It might be that some measurements are much less likely to yield small permutation p-values than others, for example because some measurements have more ties. If that is true, then the value of 0.12% is likely to underestimate the FDR.
Reference
Phipson, B., and Smyth, G. K. (2010). Permutation p-values should never be zero: calculating exact p-values when permutations are randomly drawn. Stat. Appl. Genet. Molec. Biol. Volume 9, Issue 1, Article 39. https://doi.org/10.2202/1544-6115.1585
