1
$\begingroup$

I wish to semi-randomly generate values from the known distribution of a particular quantity. The distribution is represented in my data as a list of values with a probability for each value, where the sum of all probabilities is equal to 1.0

I am using numpy.random.choice to randomly select from those discrete values based on their probabilities. I do this 20,000 times to generate a list of 20,000 values drawn from the known distribution.

Now, I want to quantify the uncertainties of each of those 20,000 values, but I am not sure how to do so because I have a poor understanding of statistics. Is the uncertainty of any given value equal to the standard deviation of the parent distribution?

$\endgroup$
3
  • 2
    $\begingroup$ What do you mean "uncertainty"? If the distribution is known, there is no uncertainty in the 20,000 values that you have generated from that distribution. The empirical distribution of the 20,000 values should be consistent with the CDF whence you've drawn the samples. That means functions of data, such as the sample standard deviation should estimate the population standard deviation (referring to the function of the DF you have). $\endgroup$
    – AdamO
    Commented Jul 14, 2015 at 22:22
  • $\begingroup$ i must have misused the word "known" in this context. the data from which the parent distribution was derived have uncertainties, eg. for x=18.5, y= 919. +/- 72.3, for x=19.0, y= 1390. +/- 145., etc. I transformed the data to a probability distribution by normalizing the histogram of the data. Thus, there is uncertainty on the probabilities of each value of x. For this reason, my 20,000 randomly generated values should take the form x=18.5 +/- 0.3 (I made that number up). Right? $\endgroup$
    – mlh
    Commented Jul 14, 2015 at 22:52
  • $\begingroup$ You'll need a model for how the uncertainty on each observation is related to the values. Since this seems to be predicated on a relationship between two variables, you will likely need to avoid combining the data into an unconditional distribution for one variable, but to keep the original information of x,y pairs (with uncertainties) and build a model for the relationship and the uncertainties.. $\endgroup$
    – Glen_b
    Commented Jul 15, 2015 at 1:30

1 Answer 1

1
$\begingroup$

There are two sources of uncertainty in the setting you have described.

First, there is sampling uncertainty: whether your sample distribution (i.e. the distribution of 20K randomly drawn values) is similar to the population distribution (i.e. your "known" distribution). By your description, your population distribution $X$ is discrete, with $k$ values in the support of $X$. If $k = 100$ and you draw $N = 10$ sample values, then it is not very likely that your sample distribution will closely match your population distribution (assuming the Pr$(x_i) >> 0$ for all $i$ in $(1,k)$ ). If you draw $N = 1000$ samples, then your sample distribution will be much closer, and so on as $N \rightarrow \infty$, thanks to the Central Limit Theorem and Law of Large Numbers. With your sample size of $N = 20,000$, the deviation is going to be quite small, and therefore sampling uncertainty will be close to zero.

Second, there is model uncertainty, which is the uncertainty associated with your "known" distribution. For discrete distributions, this can take several forms. You are using a distribution with $k$ possible values in the support, but maybe $k$ is not the right number (i.e. doesn't match reality). Maybe it's more, maybe it's less. For now, I'm going to assume that $k$ is the right number of possible values for Pr($x$).

If so, your major source of model uncertainty will be your estimates of the discrete probability for each possible value of $x$. These can arise through inaccuracy (i.e you estimate Pr($x_i) = 0.2$ when really Pr($x_i) = 0.1$), imprecision (i.e. you estimate Pr($x_i) = 0.3$ when really Pr($x_i) = 0.33333333\dots$), ambiguity (i.e. you estimate Pr($x_i) = 0.2$ when really Pr($x_i) = 0.1$ or Pr($x_i) = 0.3$), or incompleteness (i.e. you don't have an estimate for Pr($x_i$) so you interpolate using Pr($x_{i-1}$) and Pr($x_{i+1}$).

Perhaps the simplest way for you to deal with model uncertainty is to generate three population distributions. The first step is to estimate an upper limit and lower limit for each value Pr($x_i$). These should be "90% confidence" intervals. It is possible to elicit such estimated limits from experts after you have put them through a calibration exercise (see the book How To Measure Anything by D. Hubbard).

The second step is to generate the three population distributions. The first distribution you already have: your "expected population distribution". The second would be "lowest mean population distribution", where the estimates for individual values Pr($x_i$) are chosen from the upper or lower limits such that the overall distribution has the lowest mean value. Effectively, you are adjusting the probability estimate to the upper limit for the lowest values of Pr($x_i$) and the reverse for the highest values of Pr($x_i$). Then you reverse this procedure to produce the third distribution: "highest mean population distribution".

Once you have these three population distributions, you then run your simulation three times, generating 20,000 samples from each of the three distributions. You can then perform any statistical analysis you want on the resulting sample distributions (i.e. comparing sample means, sample variance, etc.) to yield an upper limit, a lower limit, and expected value.

This last step is optional, because as I said above, with 20,000 samples, the sampling uncertainty will be very low -- close to zero -- so you can do the analysis on the three population distributions themselves.


The above approach is simplified, but I offer it to you because you say you have a limited knowledge of statistics. This approach assumes that the uncertainty in estimates of each Pr($x_i$) is fairly uniform over all $i$ in $(1,k)$. If, instead, nearly all of your uncertainty is concentrated in just a few $x_i$, then you'd need to follow a different procedure -- Monte Carlo simulation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.