# How good is Monte Carlo Simulation when the variable distribution is unknown?

I am reading the book "how to measure everything", there is a chapter when the author encourages the usage of Monte Carlo simulation in simulating the future events in order to get a better understanding of the risk/ uncertainty involved. The theory sounds great in paper, and I am tempted to apply Monte Carlo techniques to startup analysis, to study the performance of employees, sales campaign, markets... everything!

But in practice, I doubt whether Monte Carlo simulation is useful when the distribution of the variables involved in an equation is unknown. Think of the stock price return distribution-- it doesn't fit into a Gaussian Distribution, but what distribution we can use? And when the variable distribution is unknown, how useful is Monte Carlo simulation? My guess is not very useful, but I am right?

A relevant, but probably narrower question is, is there any research done along the line of sensitivity analysis? By "sensitivity analysis" I mean we assume known distributions for the variables, but we perturb them a little by slightly changing the distribution shape or slightly changing some of the constants, will the outcome of the simulation results remain roughly the same, or completely different?

In practice simulation is almost always done without full certainty about the distribution.

For example, one might consider a lognormal distribution for some random variable... but what if it was a little lighter tailed (gamma, say)? Or heavier tailed (inverse gamma, say)? Or much heavier tailed (inverse Gaussian, say)? Or log-t?

What if it was some mixture of lognormals? ... or perhaps some form of regime switching?

What if there was a slight dependence in the data?

One can consider a host of ways in which the model may be inadequate (preferably springing from an understanding of the process being dealt with), and see via simulation how that affects the conclusions.

Such things are commonplace. It's one reason why simulation is so useful

Think of the stock price return distribution-- it doesn't fit into a Gaussian Distribution, but what distribution we can use?

If you read around a little you can find a variety of reasonably good models; none will be perfect.

A relevant, but probably narrower question is, is there any research done along the line of sensitivity analysis? By "sensitivity analysis" I mean we assume known distributions for the variables, but we perturb them a little by slightly changing the distribution shape or slightly changing some of the variables, will the outcome of the simulation results remain roughly the same, or completely different?

Yes, in a variety of guises (and not just for simulation; it's a standard tool in robustness for example) -- but the perturbations needn't be small; you can investigate the effect of just about any form of variation from the base assumptions.

In robustness the effect of small perturbations in distribution is considered through tools like influence functions, empirical influence functions and sensitivity curves (and also via simulation)

The first several books at the above wikipedia page are a good starting place for references on robustness; I don't recall exactly what's in which book (I read Hampel et al in the mid 80's and Huber a year or two later, and some of the others since - it's been a while), but the Hampel et al book does discuss influence functions, so I'd start there I guess.

Edit: TooTone reminded me that I wanted to mention resampling methods, which can be thought of as a kind of simulation. For example, with simple bootstrapping, you use the ECDF (the cdf of the sample) rather than the assumed distribution to sample from. In more complex resampling schemes some function of the data (such as model residuals) may be resampled. So these approaches needn't rely on some parametric assumption about the distribution the data were drawn from.

(Then again, there's also the parametric bootstrap which might be as readily thought of as a particular simulation technique as bootstrapping.)

• On the robustness thing, you have any books or articles to refer to? Commented Mar 5, 2014 at 7:21
• Glen_b, would you like to incorporate your comments into the answer? Commented Mar 5, 2014 at 8:46
• Done, and added some information that should have been obvious to me before. Commented Mar 5, 2014 at 8:52
• The Huber text wouldn't happen to be written by a certain CV moderator, would it?
– Sycorax
Commented Mar 5, 2014 at 13:28
• v good answer imo. Also is there any scope for non-parametric methods, e.g. incorporating sampling from historical data to avoid specifying the distributions? Commented Mar 5, 2014 at 18:41