Determine unknown probabilities from sample of multinomial distribution

Question

Let there be a bag with arbitrarily many balls in k colours (presume we know k). There is a set (but unknown) probability p1, p2... pk of drawing a ball of a given colour. I take a sample of g balls (replacing each ball after drawing it), and get some frequency count of each colour.

Based on this sample, how might I go about:

1. expressing my confidence/uncertainty that the observed frequencies reflect the underlying probabilities p1, p2... pk (i.e., putting error bars around the relative frequencies observed); and
2. calculating the expected value of p1, p2... pk.

I understand that this will involve a multinomial distribution, but all examples I can find go the other way round (i.e., determining the probability of making a given draw from a bag with known colour probabilities). This Wikipedia page (https://en.wikipedia.org/wiki/Checking_whether_a_coin_is_fair) provides the relevant formulas for binomial distributions, but I can't see how to extend these to multinomials.

Many thanks for any assistance!

Answer 1

To answer 1 and 2, use the Dirichlet distribution as the conjugate prior distribution before observing the data $g$. With no data, assume a uniform prior, which means initializing the distribution parameters as $\alpha_1 = \alpha_2 = \dots = \alpha_k$. Thus if $p = (p_1, \dots, p_k)$ are the prior probabilities as you denote, then $p \sim Dir(k, \alpha)$, where $\alpha = (\alpha_1, \dots, \alpha_k)$. If $(x|p) \sim Cat(k,p)$ where $Cat$ is the categorical distribution and $x = (x_1, \dots, x_k)$ is the observed data, then $(p|x, \alpha) \sim Dir(k, \alpha +c)$ where $c = (c_1, \dots, c_k)$ are the number of occurrences in the data in each category.

After obtaining the posterior distribution for $p$ with parameter $\alpha' = \alpha + c$, to express the uncertainty of each $p_i$, you can calculate $Var(p_i)$ which is given as $$ Var(p_i) = \frac {\alpha'_{i}(\alpha'_{0}-\alpha'_{i})}{{\alpha'_{0}}^{2}(\alpha'_{0}+1)}$$ where $$\alpha'_0 = \sum_{i=1}^k \alpha'_i.$$ To calculate the expected value for each $p_i$, we can use the formula $$E(p_i) = \frac{\alpha'_i}{\alpha'_0}.$$

All of this can be seen on the linked Wikipedia page.