For a simulation, I want to construct the probability distribution of a random variable $X_k$ that takes only finite number of values $x_1, \cdots, x_N$. I have to assign values to each probability: $Pr(X=x_1), \cdots, Pr(X=x_N)$, and of course they should sum to one: $\sum_{i=1}^{N} Pr(X=x_i)$. I have two ways in my mind:

1) I generate $N$ values using a continuous uniform random number generator, and then divide each by the total.

2) I generate $N-1$ values using a continuous uniform(0,1) random number generator, sort them out and treat these numbers as cut points for the probabilities.

What can I say about these two ways? Are they equivalent in the sense that the distributions of the generated probabilities are the same? If different, is there any reason to choose one over the other?

  • 3
    $\begingroup$ First, we cannot judge if one of the methods is better than the other when you did not tell us what is your objective! So, what do you want to achieve? Why do you want to do this? $\endgroup$ – kjetil b halvorsen Oct 30 '14 at 9:41
  • $\begingroup$ This question essentially asks how to generate data from a prior distribution for $X$. It is not answerable until you stipulate what the prior distribution ought to be. We can also give advice on selecting prior distributions, provided you supply guidance concerning the purpose of your study. $\endgroup$ – whuber Oct 30 '14 at 13:56

The methods are not equivalent. To see this, let us pick $N=2$, denote the random probabilities by $(P_1,P_2)$ and compute \begin{equation} \mathbb{P}\left(P_1 \leq \frac{2}{3}\right). \end{equation} for both methods.

Method 1

Method 1 in the $N=2$ case works as follows. $X_1,X_2$ are iid. $\mathrm{Uniform}(0,1)$ random variables and \begin{equation} P_1 = \frac{X_1}{X_1 + X_2} \end{equation} Therefore, \begin{equation} \mathbb{P}\left(P_1 \leq \frac{2}{3}\right) = \mathbb{P}\left(\frac{X_1}{X_1+X_2} \right) = \mathbb{P}\left(X_1 \leq 2X_2 \right). \end{equation} There are various straightforward ways to evaluate the last expression. For example, $(X_1,X_2)$ is uniform in the unit square $[0,1]\times[0,1]$, so the probability is simply the area where $X_1\leq X_2$. Alternatively, let us decompose it into two conditional probabilities conditioning on whether $X_2\leq\frac{1}{2}$: \begin{equation} =\mathbb{P}(X_2\leq \frac{1}{2}) \mathbb{P}\left(X_1 \leq 2X_2 \mid X_2 \leq \frac{1}{2}\right) + \mathbb{P}(X_2 > \frac{1}{2}) \mathbb{P}\left(X_1 \leq 2X_2 \mid X_2 > \frac{1}{2}\right) \end{equation}. Conditional on $X_2 \leq\frac{1}{2}$, $2X_2$ is uniform in $[0,1]$ and thus for symmetry reasons $\mathbb{P}\left(X_1 \leq 2X_2 \mid X_2 \leq \frac{1}{2}\right) = \frac{1}{2}$. If $X_2 > \frac{1}{2}$, $2X_2 > 1$ and therefore $X_1 < X_2$. Thus $\mathbb{P}\left(X_1 \leq 2X_2 \mid X_2 > \frac{1}{2}\right) = 1$. Finally, $\mathbb{P}(X_2\leq \frac{1}{2}) = \mathbb{P}(X_2 > \frac{1}{2}) = \frac{1}{2}$. Plugging these values into the expression, we get \begin{equation} = \frac{1}{2}\times \frac{1}{2} + \frac{1}{2} \times 1 = \frac{3}{4}. \end{equation}

Method 2

In the $N=2$ case there is only one cutpoint uniform in $[0,1]$ and this cutpoint directly equals $P_1$. Thus $P_1 \sim \mathrm{Uniform}(0,1)$ and \begin{equation} \mathbb{P}\left(P_1 \leq \frac{2}{3} \right) = \frac{2}{3}, \end{equation} which is not equal to the value obtained with Method 1.

Which one to use

As kjetil b halvorsen pointed out in comments, it is impossible to judge which is better for your use case without knowing more details about your use case. Besides these approaches, you may also want to investigate Dirichlet distribution which is often used in Bayesian statistics to define probability distributions over discrete probability distributions (just as the methods described in the question). The Dirichlet distribution has $N$ parameters which you could tune to obtain distributions closer to desired behavior. On the other hand, the use of the Dirichlet distribution is mainly motivated by its useful computational properties in the Bayesian context, which may be totally irrelevant in your use case.

Another suggestion is to do your simulations using various distributions over the probabilities. If the conclusions you are interested in do not change, good, if they change, you need to think more about which distribution you should use.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.