5
$\begingroup$

I have a complex distribution which I can numerically sample.

I'd like to estimate a percentile (let's say 90%) using Monte-Carlo simulations. What I'm doing is:

  1. I run 1 million of independent simulations and I get 1 million of samples from the distribution.
  2. I order them in ascending order, and I take the one that it's in the 900000th position.

Assuming that this way of proceeding is correct, how can I theoretically estimate the number of runs so that the error in the estimation is roughly within i.e. 1%?

All I can say about the underlying distribution is that it's roughly analogous to a normal distribution divided by a chi distribution, if that's of any help.

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ This is just a complicated way of asking about binomial proportion confidence intervals, so you are likely to find useful information in the linked search. $\endgroup$
    – whuber ♦
    Dec 24, 2016 at 15:03
  • $\begingroup$ Why don't you edit this again taking out what is irrelevant and make it really clear what your problem is. $\endgroup$
    – Michael R. Chernick
    Dec 24, 2016 at 20:46
  • $\begingroup$ @MichaelChernick If I knew that those things were irrelevant I wouldn't have added them. By the way, I believe that what I'm asking is quite clear: how can I estimate the number of samples to use to estimate the inverse CDF at a given percentile within a given accuracy? $\endgroup$
    – xanz
    Dec 24, 2016 at 21:46
  • 1
    $\begingroup$ The question is rather vague, you don't seem to know the distribution, or it is so complicated that working with it analytically is not possible anyway. In these circumstances, why not just ... Do your simulation 100 times with $n = 10^6$ ... and see if the bulk (say 95%) of the answers lie within a 1% range (or whatever you require). If they do not, increase the number of simulations to $n=10^7$, and try again. If not, try $n=10^8$, etc If you never get there, you may need to relax your requirements. ........ $\endgroup$
    – wolfies
    Dec 25, 2016 at 16:45
  • 3
    $\begingroup$ Thank you for the clarification. You are asking for a tolerance interval for $F$. $\endgroup$
    – whuber ♦
    Dec 25, 2016 at 23:34

1 Answer 1

Reset to default
1
$\begingroup$

If the random variable under consideration is approximately the ratio of a normal to a chi, then you may just be able to use the t-distribution with appropriate degrees of freedom to approximate the quantile of interest.

If $Z \sim N(0, 1)$ and $Y \sim \chi^2_n$ and if Z and Y are independent, then it is well known that the ratio $T = \frac{Z}{\sqrt{Y/n}} \sim t_n$. If you approximately have the ratio of a normal to a chi with n degrees of freedom (independent), then it sounds like you approximately have a scaled t distribution. You can use properties of the t to estimate the quantile.

Improve this answer
$\endgroup$
3
  • $\begingroup$ They are not independent, unfortunately $\endgroup$
    – xanz
    Dec 24, 2016 at 21:42
  • 1
    $\begingroup$ @freik wrote "They are not independent, unfortunately." Kind of an important detail to have left out of your statement "All I can say about the underlying distribution is that it's roughly analogous to a normal distribution divided by a chi distribution, if that's of any help." $\endgroup$
    – Mark L. Stone
    Dec 24, 2016 at 23:46
  • $\begingroup$ I said "roughly" on purpouse because the degree of dependency is very modest. But the accuracy I require on the estimates is very high. That's why I didn't add details, I wanted to know if there's a method that's generally valid for choosing $n$ regardless of the underlying distribution $\endgroup$
    – xanz
    Dec 25, 2016 at 8:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.