How to interpret False Discovery Rate?

I have performed a time series analysis and I have a dataframe of p-values. I want to performe a False Discovery Rate analysis to account for type I error. I am an R user and I have the following.

library("astsa")

FDR(as.vector(example), qlevel = 0.05)


It is the first time using this statistic and I am quite confused. How can I know if my FDR is good? What is and appropriate value and how can I interpret it?

Reading the code and vignette for FDR from astsa, it seems like it is a lift from this link and they modified it to return the number of rejects.

If we use the code they lifted from, it is a benjamini-hochberg correction:

fdr.basic <- function(pvals,qlevel=0.05){
n <- length(pvals)
sorted.pvals <- sort(pvals)
sort.index <- order(pvals)
indices <- (1:n)*(sorted.pvals<=qlevel*(1:n)/n)
num.reject <- max(indices)
if(num.reject){
indices <- 1:num.reject
return(sort(sort.index[indices]))
} else{
return(NULL)
}
}


And compare it against the commonly used function p.adjust, where we set an FDR of 0.05:

library(astsa)
sum(p.adjust(pv$pvals,"BH")<0.05) FALSE TRUE 450 1650 length(fdr.basic(pv$pvals,0.05))
[1] 1650

astsa::FDR(pv\$pvals)
[1] 1


And you can see the function from astsa gives you something that makes no sense. I suggest you keep to using p.adjust(..,method="BH").

To answer your question, if you set an FDR of 0.05, you expect the proportion of "false discoveries" (rejected null hypotheses that are incorrect rejections) to be 0.05. So in this example if your get 1650 hits with an FDR of 0.05, you can estimate the number of false discoveries to be around 1650*0.05 = 82.5.

It depends on the purpose of your hypothesis testing. If let's say it's for a publication and you want to show you have controlled for multiple testing, the result above is ok, it shows you have an enrichment of so called positive hits.

If you want to come up with a list of good confident or positive hits, then you can go lower with the FDR and see which level suits your purpose best. You can also check out discussions such as this for definition of FDR, FDR for researchers.

• thank you for your reply and the suggested sources! Mar 30, 2020 at 14:09
• you're welcome :) glad it's useful ! Mar 30, 2020 at 15:27