I prefer to talk here and not in the ~comments~ because it seems to be a long writing. I am not a statistician on discipline :) but here are some proposal aspect for setting up your MC analysis...
Number of Experiments
OF course, the more experiments you try, the better, so, by far, the preferable Procedure would be the first. The ideal would be to distribute as many experiments as possible, because every experiment drive a different operating point of the whole system. Note that the number of points are distributed after the number of experiments were obtained...
The main factors on this decision are:
- The variability of the expected value you want to obtain,
- The variability of the process,
- The quantity of excited subsystems.
Of course, you can escalate this strategy onto sensors, machines, process, facilities, etc... The idea is, the size of the MC is theoretically the variabilities described above.
Here, variability is: non-linearity, dinamical, time-varying.
Evidently if everything is linear -this do not means the systems are linear , but their frequent operating domain is reduced to a small quantity of points- there is no difference between the Procedure 1 and the Procedure 2.
- A non-linear system-i.e. a pressure control system- or machine will require more MC points than a linear one -i.e. a temperature control system-,
- A dynamical system -i.e. almost any process- will require more MC points than a static system -i.e. a system output, constant KPIs, monitors-,
- A time varying system -i.e. event ocurrences, periodic or non periodic trends- will require specific MC points for covering those situations.
Every of these factors carry a particular, well defined, measure of variability as the expected rate of change -flow, energy consumed, activity- per rate of resources consumed -time, fuel, manhours-.
With all this clear, you are free to proceed onto the stages of the overall system analysis, which more or less proceed on this way:
Step 1. Exploration. You dont know anything about your system, and place your MC 100k operating points randomly and you are done,
Step 2. Detection. You need to identify how to distribute those 100k, in order to concentrate your simulation power to cover the most demanded areas,
Step 3. Diagnosis. You obtain reliable estimators and calculate the first variance for your estimators,
Step 4. Isolation. You optimize your simulations and improve the variance for your estimators.
Every system of course will have its own agenda, but as you see, the more complex, the merrier the analysis turned to be...
Well if any comments, just write here below and we extend the discussion.