I feel I'm pretty new to this, since some time passed since my last statistics assignment, so please bear with me.
I am analyzing results of a biological experiment. Basically, I'm looking at a some graph over a genome, where each position in the genome has a value, and I'm looking for local minima (peaks).
Now, I have to set some threshold since relatively high local minima also occur by chance. I can simulate the experiment computationally and get new data, but this is quite resource demanding (I can't run 1000 simulations, perhaps 100 or even only 20).
What I'm currently doing is run some simulations; for each simulation: find all local minima, build a CDF for the local minima values. I then average all the simulation CDFs over all simulations so I have one 'average' CDF (CDF_simulations) which is supposed to show how would local minima distribute if everything in my genome is random.
I do the same for the real data: find minima and build CDFs for their values, so I now have two CDFs - one for the real data and one for the average of the simulations.
I now search for the max x such that CDF_simulations(x) / CDF_realdata(x) is < 10%. I report all minima in the real data with value < x as "true".
I think this method should get me to a FP rate of 10%.
Does this make sense?
What is this method called and where can I find more about it?
How should I scan the CDFs to find the right x? Sometimes for low x's CDF_simulations(x) > CDF_realdata(x).
Where does the number of simulations come into play? Does it make sense to simply build an averaged CDF as I did?
I think this is quite common, and the name FDR also pops to mind, but when reading about FDR I couldn't exactly figure how to apply it to my situation.
Any comments and references (hopefully user-friendly ones) will be appreciated!
Thanks Dave