7
$\begingroup$

This may sound like a noob question but I'm unable to find any 'good' resources/examples on the same. The basic question is this: Most variables, depending on the problem will follow certain types of distributions. Normal/Gaussian may not be the most appropriate one for capturing certain types of phenomena.

Although I'm quite familiar with various distributions from a mathematical viewpoint I'm unable to understand some of them conceptually e.g.: Uniform distribution is when the occurrence of that event is equally likely over time, Normal when the occurrence 'tends' to be centered around the mean more often (like number of defects in samples or heights of citizens in a country etc.,) Similarly for triangular - I understand these easy ones so to speak.

What type of distributions have you commonly encountered when using monte-carlo simulations? Examples would be helpful along with the rationale for choosing that distribution. Basically looking for a reference/pointer that would help me lay it out as a list for reference and understanding. I'd prefer a non-mathematical explanation since it'll be used for discussing with non-mathematical stakeholders to whom the monte-carlo simulations would be shown

  • <"Distribution name"> : <"Most appropriate use">

I've heard of the power law but don't really know/understand what it is and how it could be used.

Improve this question
$\endgroup$
7
  • $\begingroup$ Your question is pretty vague; some context about what sort of process you're trying to simulate would be very helpful. $\endgroup$
    – JMS
    May 2, 2011 at 4:27
  • $\begingroup$ @JMS: Well I'm just looking for the various distributions so I can study their relevance. Basically when confronted with a problem I wish to choose an appropriate distribution. Specifics: I'm looking to perform risk analysis in a software project using monte-carlo distributions. The type of variable I may need to model is very context dependent e.g.: Risk of the project being called off midway through development - how best to capture this as a distribution to model it? There could be others, so I'm just looking for a layman's descriptions of different type of distributions... $\endgroup$
    – PhD
    May 2, 2011 at 4:39
  • $\begingroup$ Your question remains too vague IMO. Any good introductory probability text (Sheldon Ross's or Jim Pitman's, for example) will give not only mathematical definitions of probability distributions but also their properties and physical processes which give rise to them. You will almost certainly learn more distribution theory there than here. $\endgroup$
    – JMS
    May 2, 2011 at 4:56
  • $\begingroup$ @JMS: Agreed. I was more keen on knowing what other applicable distributions have people tried and if I could gleam anything from their experience/suggestions. I'll look up the books you suggested for sure. However, if the moderators deem this question to be too vague and would like to close it, I'd accept the decision! $\endgroup$
    – PhD
    May 2, 2011 at 5:04
  • $\begingroup$ Oh I wouldn't vote to close; I just thought with some extra information we might be able to help more. Hopefully you've got a good jumping off point now :) $\endgroup$
    – JMS
    May 2, 2011 at 14:43

4 Answers 4

Reset to default
4
$\begingroup$

The books "Continuous univariate distributions" Vol 1 + Vol 2 by Johnson, Kotz and Balakrishnan (and there is a multivariate book too, I believe) are classical references, rich on the mathematical properties as well as giving examples of the usages of the different distributions they treat.

If you want details on a specific class of distributions, Wikipedia is always a good place to start, see

http://en.wikipedia.org/wiki/Power_law

The requested list is probably not easy to compile - the "most appropriate use" may be highly dependent upon context, but again the Wikipedia list of distributions

http://en.wikipedia.org/wiki/List_of_probability_distributions

could be a place to start to find distributions appropriate for your project.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Great! I looked through the contents and a sample chapter and it seems under the application section for each distribution it lists where/how/why was this distribution chosen and to model what phenomena. That's pretty much what I was looking for. I've already gone through the wikipedia links :) I guess I should be able to take it from here - else the forum is always there to help with the clarifications :D $\endgroup$
    – PhD
    May 2, 2011 at 6:30
3
$\begingroup$

I personally worry about the philosophy of deciding on a distribution with certain properties first and then defining the parameters for the distribution.

My advice is normally to get the best data that you can first and then to process that data to find the best distribution and parameters to fit it. If you decide a distribution first to suit the input data then you are deciding on the way the data should behave before you have actually checked that it behaves how you have assumed in reality. A simple example would be a data set where you assume that all of the variations are due to measurement errors, in which case a normal distribution would probably be your chosen distribution. What you can often find though is that the data is not normally distributed and there is something more complex underlying the data you have.

One of the biggest problems in the modern use of Monte-Carlo analysis is the use of assumed distributions based on the best guesses of the users rather than actual data. If you don't actually have the data on which to estimate the input distributions then I would argue that there are better ways of looking at the problem than just going for Monte-Carlo analysis in the first instance.

Improve this answer
$\endgroup$
6
  • 2
    $\begingroup$ The problem isn't picking a model beforehand, it's not doing the model checking afterwards! If you just pick a model to fit your data, without using any substantive knowledge of the problem, you're all but guaranteed to overfit it. Or you might miss important features that aren't present in your dataset (heavy tails, for example) A better "workflow" is 1) Pick a model that may reasonably represent your problem 2) Fit the model to the data 3) Check the model 4) Rinse and repeat. $\endgroup$
    – JMS
    May 2, 2011 at 14:40
  • $\begingroup$ @Ian: That's a great point. My only concern is that sometimes we really don't have any input data and have to model it with certain assumptions. Like the example I gave in a comment above - the risk of a project being called off midway. Past history may be only 2-3 data points but to 'estimate' risk in this case would require a 'model fit' as in @JMS step 2. I agree steps 3 and 4 are equally important! $\endgroup$
    – PhD
    May 2, 2011 at 17:00
  • $\begingroup$ @Ian: You mention that ...there are better ways of looking at the problem... Could you please throw more light on that? $\endgroup$
    – PhD
    May 2, 2011 at 17:00
  • $\begingroup$ @Nupul Do you really only have 2 or the data points? What about thinking in terms of Pr(project not called off)? Also, Ian is talking about fitting a model too but his answer (the way I read it) would have you get some data, pick a model based on that data, and then fit that model to said data. This is overfit city; you can always find a model to fit your data arbitrarily well, but it won't perform out of sample. $\endgroup$
    – JMS
    May 2, 2011 at 18:29
  • 1
    $\begingroup$ @nupul regarding the other ways to look at the problem, the issue with Monte-Carlo analysis is that it is prone to hide the fact that you don't understand the inputs very well under the blanket of sophisticated analysis. For this reason it is a good tool by those trying to 'hide' a lack of input data or understanding. For this reason I would recommend simple parametric analyses on key variables initially to understand how they affect the model and then report on those results rather than hiding it behind a complex Monte-Carlo analysis. $\endgroup$
    – Ian Turner
    May 2, 2011 at 22:02
2
$\begingroup$

I know I am resurrecting an old thread but I had recently been looking for similar information and was unable to find it on the forum. I eventually managed to implement a solution for my study and I will add my experiences on using Monte Carlo simulation.

Monte Carlo Simulations is a rather broad, all-encompassing term for any statistical simulation method involving random sampling performed a large number of times. An important point to note about this method is that, in general, it will not lead to an improved estimate of the desired statistical property (the mean, for example). What it does provide however, is a means to quantify the uncertainty in that estimate.

Your question could be a little bit misleading since Monte Carlo simulations are not only used to simulate known distributions - in fact one of the strengths (and an oft encountered use-case) of this method is when one does not know the population distribution. While the empirical density distribution of one's data can be approximated by theoretical distributions for some well-known phenomena, more often than not this is not the case. In such situations, it is recommended to instead use the Monte Carlo bootstrap which can be found to be explained well in Michael Chernick's book among others.

The algorithm for implementing this is:

For i = 1 to nsims
    Do   
      Sample = Sampling with replacement ();
      Calculate_Mean;
      Calculate_Standard_Deviation;
      Calculate_Variance;
      Calculate_Probability_of_Mean_exceeding_a_threshold;
      _etc..._
    End Do
End For

I implemented this in R for my study, after having explored the possibility of fitting known distributions and mixture models (my related questions on the same can be found here and here). I was doing a meta-analysis of data values on energy demand obtained from literature. I was working with small sample sizes (between 20 and 50) and wanted to come up with the uncertainty estimates for the mean and the probabilities of the mean exceeding certain thresholds. I cannot yet publish the results here but I can perhaps update this post at a later time.

This does not directly answer the OP's question, but to sum up, in general my advice is to:
1. check if your data matches some known distribution the sources for which have already been listed in other answers
2. Simulate and map the uncertainties and
3. repeat the same with a Monte Carlo bootstrap.

If the results are more or less the same then you have a fairly robust estimate. If they are very different, I would advise to go with the results from the Monte Carlo bootstrap since they do not impose any a-priori assumptions.

Improve this answer
$\endgroup$
1
1
$\begingroup$

If you know the domain of the random variable and maybe have knowledge of some other properties like the mean, variance, etc. but want to be ignorant in a fair way about all other aspects of the distribution, you can find a distribution by applying the principle of maximum entropy. Or put simply

Given a collection of facts, choose a model which is consistent with all the facts, but otherwise as uniform as possible. (Adam Berger)

Even if you don't want to derive these distributions yourself i think it's still good to know that there is such a general principle that may be used. For many common cases people have already done this and you can just look up the solution, depending on your domain and given statistics: Wikipedia lists some of them.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.