The “simulation” approach to probabilities, where you have a model and you simply sample sufficiently many points from a distribution to (hopefully) estimate your probability well enough, is encountered everywhere in computer science.

The issue that I have with this approach is: Once you have the randomly sampled data points, there is no mathematical way of connecting them back to the distribution they came from. They are just a sequence of numbers (if you have many of them, you can test them for randomness, but that is really all you can do - and even that test will give you an answer that has a probability of it being true attached to it).

How do you go back from the list of numbers to the distribution that generated it?

  • 1
    $\begingroup$ It's not entirely clear to me what you mean, but you might be interested in looking at things like the empirical process and the empirical cdf. Here you take random samples to make an empirical estimate of the distribution function and the Glivenko-Cantelli theorem tells us that if our samples are iid this empirical cdf convergences functionally to the underlying distribution F as the sample points goes to infinity non-parametrically. This kind of result is used often formally in machine learning and statistical learning theory, see chapter 3 econ.upf.edu/~lugosi/mlss_slt.pdf $\endgroup$ – Tyrel Stokes Aug 29 at 17:41
  • $\begingroup$ what do you mean by mathematical way of connecting back? if you forgot what the source distribution was, but had some candidate distributions in mind, you could still identify which distribution the samples were drawn from by matching properties of the data, such as its higher order moments, with those same properties of one of the candidate distributions $\endgroup$ – develarist Aug 29 at 17:43
  • $\begingroup$ @TyrelStokes Thanks, this was a really interesting pdf to read. I'm so unsure about the whole topic that it is even hard for me to precisely formulate what I am unsure about - but what you mentioned definitely points into the right direction. $\endgroup$ – user52145 Aug 29 at 18:29
  • $\begingroup$ @develarist Really, is that possible? Can you elaborate on that a bit, please? Or share some links? $\endgroup$ – user52145 Aug 29 at 18:31
  • $\begingroup$ Could you give example of what do you mean? $\endgroup$ – Tim Aug 29 at 19:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.