1. From Applied Statistical Genetics with R by Foulkes (pFDR = positive false discovery rate):

    The q-value is defined similarly as the greatest lower bound of the $pFDR$ that can result from rejecting a test statistic $T$ and a sample $x$ with $T(x)=t$ based on a rejection region $Γ$ in the set of nested rejection regions. Formally, the q-value is written $$q(t) =\inf_{Γ :t∈Γ } [pFDR(Γ )]$$

    Since $pFDR$ is involved here, there must be several null hypotheses, and different null may have different testing rule. But here does it assume that there is one common testing rule $(T, Γ)$ for all the nulls?

  2. From Wikipedia

    The q-value is defined to be the FDR analogue of the p-value. The q-value of an individual hypothesis test is the maximum FDR at which the test may be called significant. One approach is to directly estimate q-values rather than fixing a level at which to control the FDR.

    • Why are we talking about the q-value of an individual hypothesis test? Isn't the q-value for multiple nulls and their test rules?

    • Should "maximum FDR" be "minimum FDR" instead?

    • What does "One approach is to directly estimate q-values rather than fixing a level at which to control the FDR" try to mean? What is "one approach" trying to solve?

  3. Am I right that pFDR depends on and changes with the sample's distribution, so does q-value?

Thanks and regards!

  1. Indeed. It is assumed that all test statistics that fall in $\Gamma$ are rejected and others are not. Also note that in a pure (i.e., subjective) Bayesian framework, there need not be multiple hypotheses to define the pFDR. This is, however, now the view Storey has been promoting.

  2. This Wikipedia entry actually discusses a different meaning/definition of the q-value, which preceded Storey's definition and makes life rather confusing (see also here). The "individual" hypothesis discussed is the marginal hypothesis rejected. It is true that hypothesis will typically not be rejected alone. That is why the marginal rejection is discussed. It should indeed be the minimal FDR. The two competing approaches to multiplicity control are: (1) Set an error level and find a procedure that controls it. (2) Estimate the error level and reject when the estimated error level reaches a desired value.

  3. The estimated $pFDR(\Gamma)$ depends on the sample. It's value does not depend on the sampling distribution of each test statistic (it is simple arithmetics), but it's sampling properties will naturally depend on the sampling distribution. Particularly, the test statistics need to be independent for Storey's estimator to be unbiased to the true $pFDR(\Gamma)$.

  • $\begingroup$ Thanks! pFDR is defined in terms of expectation wrt the true sample distribution, but not directly depend on the sample? $\endgroup$ – Tim Jul 8 '13 at 11:06
  • $\begingroup$ @Tim: I distinguish between the estimated pFDR and the true pFDR. The true pFDR of any rejection region will surely depend on the sampling distribution. $\endgroup$ – JohnRos Jul 8 '13 at 12:00
  • $\begingroup$ In the first definition, is the q-value defined for each marginal null, or for all the marginal nulls in the multiple comparision? $\endgroup$ – Tim Jul 8 '13 at 23:45
  • $\begingroup$ Also in the first definition, is $T$ the test statistic of just one of the multiple comparisons? If yes, does the definition of q-value not require other comparisons's test statistics to be rejected? $\endgroup$ – Tim Jul 9 '13 at 1:01
  • $\begingroup$ Also- have a look at this manuscript of mine for a quick intro to error rates: arxiv.org/abs/1304.4920 $\endgroup$ – JohnRos Jul 9 '13 at 7:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.