1
$\begingroup$

As an engineer trying to learn statistics, I wonder if someone could please recommend references / a statistical method that may assist with determining the number of simulations that need to be completed to give confidence in the conclusions of the data analysis.

In brief, I'm running simulations to model a system, where the simulation output is an array of numbers describing some of the events that occur during the simulation.

e.g. one simulation might return (20, 21, 18, 20) and when run again (each time the simulation is run, it is completely independent), the output is (20, 18, 2)

I want to perform a global analysis on the problem I'm modelling by plotting a histogram of the values returned from the simulations. Having done this, I see roughly a log distribution (which is roughly what I expect) and some strong outliers.

Considering such outliers, the system could perhaps be deemed to display "extreme-value statistics".

However, for confidence and to populate a sufficiently-detailed histogram, I understand that the simulation must be run many times.

Is there a statistical way to calculate the number of times the simulation should be run?

Any pointers / references / help would be greatly appreciated.

Many thanks!

Improve this question
$\endgroup$
2
  • $\begingroup$ Welcome. Your criteria would need to be more specific than to have "confidence in the conclusions" and to obtain a histogram that is a "sufficiently detailed." (What constitutes "confidence" or "sufficient" for you, or for your audience?) I also think to get a helpful answer you would need to share more info about your simulation and a plot of your distribution so far. $\endgroup$
    – rolando2
    Commented Apr 1, 2015 at 23:40
  • $\begingroup$ It sounds like you want to use re-sampling to put error-bars around your quantiles. Error in the mean is textbook, but error in the 95th percentile is ... slightly less so. You want to make sure your sample size is sufficient to reduce your 95% CI on the estimates, to reduce your error bars to reasonable levels. The requirement there is that you know something about the nature of the generating distribution. Can you tell a little more about the nature of the distribution - enough to make an informative "toy" problem to show example solution? $\endgroup$
    – EngrStudent
    Commented Jul 7, 2015 at 2:29

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.