0
$\begingroup$

We want to find posterior probability mass function for dice tossing with uniform prior.

We are interested in rolling of weighted dice. The outcome is 1,2,...,6. We assume that prior probability mass function is uniform (all 1/6).

Assume that we observed 10 samples (dice tosses). Say 6 were 1 and 2 were 2 and 2 were 3. Outcomes 4 5 6 were not observed.

As far as we know, the Bayesian posterior for this case is Dirichlet(6+1,2+1,2+1,1,1,1)

Now by using code in website https://people.sc.fsu.edu/~jburkardt/m_src/asa266/asa266.html

We get dirichlet_mean(6,[6+1 2+1 2+1 1 1 1]) = [ 0.4375 0.1875 0.1875 0.0625 0.0625 0.0625]

So Bayesian posterior estimation would be exactly the same as histogram calculation, with only difference being the initial values of 1 1 1 1 1 1.

Is this correct? I suppose that if are more confident in prior probability mass function then initial values could be for example [10 10 10 10 10 10]?

We do not understand what the randomness in the dirichlet means. How can posterior probability mass function be random?

By using code in the above website we can get for example as dirichlet_sample (depends on the seed) [0.6816 0.1176 0.0271 0.0079 0.1077 0.0581]

Its roughly similar to the dirichlet mean but not the same.

Our question is that what is this randomness?

Or is the solution for posterior probability mass function simply the dirichlet_mean?

Thanks for all answers/comments!

Edit: pdf = > probability mass function / pmf

$\endgroup$
  • $\begingroup$ This data set does not come from a continuous distribution. so you must be referring to a probability mass function rather than a pdf. $\endgroup$ – Michael R. Chernick Jan 17 '19 at 19:56
  • 1
    $\begingroup$ @MichaelChernick: this is the posterior distribution on the probabilities of the six sides. $\endgroup$ – Xi'an Jan 17 '19 at 21:06
  • $\begingroup$ Okay I missed what your hypothesis was. $\endgroup$ – Michael R. Chernick Jan 17 '19 at 22:11
1
$\begingroup$

It seems that you are confusing the pmf of the multinomial random variable $X$ given a parameter vector $\boldsymbol\pi:=\{\pi_1,\pi_2,\pi_3,\pi_4,\pi_5,\pi_6\}$ with the prior pdf on the parameter vector $\boldsymbol\pi$ itself.

Your assumptions give that the pmf of $X$ given $\boldsymbol\pi$ is $\Pr(X=k|\boldsymbol\pi)=\pi_k$, with $k=1,\ldots,6$. There is nothing random about $\Pr(X=k|\boldsymbol\pi)=\pi_k$ itself, it is just a function. In fact, if you are thinking like a Bayesian, $X$ itself is usually no longer considered random, since it has been observed at the point you are fitting the model. Usually $\Pr(X=k|\boldsymbol\pi)=\pi_k$ is called the 'likelihood', and this term is used for both continuous or discrete $X$s.

Rather, it is the probability vector $\boldsymbol\pi$ that is random in this Bayesian context. The Bayesian perspective is to update some base knowledge about $\boldsymbol\pi$, which takes the form of a prior distribution, with the additional knowledge we learn from observing $X$. The $\pi_k$s are continuous but subject to the constraint $\pi_k\geq 0$ for all $k$ and $\sum_k \pi_k = 1$. The Dirichlet distribution a natural choice for the prior distribution because it enforces this constraint and for reasons of 'conjugacy' (https://en.wikipedia.org/wiki/Conjugate_prior).

The Dirichlet distribution is given by $p(\boldsymbol\pi)\propto \prod_{k=1}^6\pi_k^{\alpha_k-1}$, with $\alpha_k>0$. (see https://en.wikipedia.org/wiki/Dirichlet_distribution). It seems that the software you linked to uses a nonstandard parametrization of the Dirichlet distribution, since it requires that the $\alpha_k$s be

positive quantities which are required to sum to 1.

(https://people.sc.fsu.edu/~jburkardt/m_src/asa266/asa266.html) whereas the typical parametrization requires that $\alpha_k>0$. This may be one source of your confusion. We're now equipped to address your questions.

We get dirichlet_mean(6,[6+1 2+1 2+1 1 1 1]) = [ 0.4375 0.1875 0.1875 0.0625 0.0625 0.0625]

Yes, this is the mean of the posterior distribution of $\boldsymbol \pi$ after observing your ten dice rolls, and under a Dirichlet prior on $\boldsymbol\pi$ having $\alpha_k=1$

Is this correct? I suppose that if are more confident in prior probability mass function then initial values could be for example [10 10 10 10 10 10]?

Yes, this is possible. Your prior distribution presumably reflects your prior knowledge about the data generating system. In this case, there is an easy and intuitive interpretation: each $\alpha_k$ corresponds to a prior number of rolls showing side $k$ out of a $\sum_{k=1}^6\alpha_k$ total prior rolls, e.g. from some previous experiment.

We do not understand what the randomness in the dirichlet means. How can posterior probability mass function be random?

This is a posterior distribution function, or just the posterior distribution, of $\boldsymbol\pi$. Don't confuse the inherent uncertainty (what I believe you are calling randomness) in rolling dice with the uncertainty about the underlying parameters. Even if you rolled a million $X$s, there would still be uncertainty in predicting the million+1th roll. On the other hand, you would have near perfect certainty about the underlying probabilities of that roll being 1, 2, 3, 4, 5, or 6. In other words, you are nearly 100% certain that $\pi_1 = 0.6$ (for example), but that doesn't mean you know $X$ will be 1 next time. And you won't know any better with a million more rolls.

By using code in the above website we can get for example as dirichlet_sample (depends on the seed) [0.6816 0.1176 0.0271 0.0079 0.1077 0.0581]

This is highlighting the second type of uncertainty I described above -- uncertainty about $\boldsymbol \pi$. You have sampled from the posterior distribution of $\boldsymbol\pi$ itself. If you did this a large number of times and calculated the element-wise average, you would get [ 0.4375 0.1875 0.1875 0.0625 0.0625 0.0625]. However, continuing my example above, if you plugged in a hypothetical set of dice rolls of [600000, 200000, 200000, 0, 0] and compared the dirichlet mean to individual draws using the dirichlet sampler, you will see that they are both nearly identical. Thus, you are very confident that $\pi_1=0.6$, $\pi_2=0.2$, etc...

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.