False discovery rate (FDR) is defined as

FDR = FP / (TP + FP).

Does it estimate some population quantity, independent of sample, when the sample satisfies some condition?

To make it clearer, True positive rate is defined as

TPR = TP/ (TP + FN)

If all the points in the sample come from a sample distribution $F$ in the alternative hypothesis, true positive rate estimates $P_{X \sim F}($ reject null $)$.

  • $\begingroup$ How is the TPR related to the question? $\endgroup$
    – JohnRos
    May 19 '13 at 6:08
  • $\begingroup$ @JohnRos: If all the true positives in a sample comes from an alternative distribution $F$, TPR can be seen as an estimate of $P_{X\sim F}(\text{ reject null })$. But for FDR, because its denominator is the reported positives which is not population dependent, I don't think it can be estimate of any population quantity. $\endgroup$
    – Tim
    May 19 '13 at 9:13
  • $\begingroup$ I see. I still feel the presence of FTP might be more confusing than clarifying. $\endgroup$
    – JohnRos
    May 19 '13 at 12:06

The FDR of an inference procedure cannot estimate any quantity, as it is, by itself, a fixed quantity. Think of it as a generalization of the type I error-- which you would never call "an estimate".

One can however, estimate the FDR of a given inference procedure. Try [1] as a starting reference.

[1] Storey, John D. "A direct approach to false discovery rates." Journal of the Royal Statistical Society: Series B (Statistical Methodology) 64.3 (2002): 479-498.


Efron and Tibshirani's new book Computer Age Statistical Inference offers an intuitive understanding to what is being estimated. Let me summarize their message graphically.

The (Bayesian) underlying idea is that observations come from a mixture of two distributions:

  • $\pi_0 \: N$ observations from the null density $f_0(z)$
  • $(1-\pi_0) \: N$ observations from alternative density $f_1(z)$.

What is observed is the mixture of those two:

  • $f(z) = \pi_0 \cdot f_0(z) + (1-\pi_0) \cdot f_1(z)$

enter image description here

The (Bayesian) definitions are:

  • $\text{Fdr} = \frac{\pi_0 \: (1-F_0(z_0))}{(1-F(z))}$ (a fraction of the tail areas)
  • $\text{fdr} = \frac{\pi_0 \: f_0(z_0)}{f(z)}$ (a fraction of the tail densities)

As shown below, Fdr is equivalent to the Benjamini hocherg FDR when $\pi_0 \approx 1$ (which is the case in most bioinformatics studies)

enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.