How to estimate biases from coin and dice using only observed dice throws in this setup? To help me understand some concepts I'm learning in my first exposition to machine learning, I'm trying to tackle the following "simple" problem
The setup of the problem is as follows: 


*

*My friend has a coin and two 6-sided dice (all possibly biased)

*He first tosses a coin, if the result is heads, he throws dice A, otherwise he throws dice B

*He repeats this $n$ times and gives me the results of each die throw (without telling me which die generated which result)


How can I estimate the bias of the coin and the bias of each die using only the data he gives me?

My work so far:


*

*Bias of the coin: $\theta$

*Bias of die $k$: $\theta^k$ (6-dimensional vector)

*Bias of side $i$ of die $k$: $\theta_i^k$ (e.g. the probability of landing a 3 with die A is $\theta_3^A$)

*$i$'th unobserved coin toss result (with value either A or B): $z_i$

*All the coin tosses results: $\mathbf{z}$

*$i$'th dice throw result: $w_i$

*All the dice throws results: $\mathbf{w}$ 

*I want to compute $P(\theta,\theta^A,\theta^B~|~\mathbf{w})$. To do that, I use bayes' rule:


$$ P(\theta,\theta^A,\theta^B~|~\mathbf{w}) = \frac{P(\mathbf{w}~|~\theta,\theta^A,\theta^B)\cdot P(\theta,\theta^A,\theta^B)}{P(\mathbf{w})}$$


*

*To compute $P(\mathbf{w}~|~\theta,\theta^A,\theta^B)$ I first note that the throws are independent of each other, so 


$$P(\mathbf{w}~|~\theta,\theta^A,\theta^B)=\prod_{i=1}^nP(w_i~|~\theta,\theta^A,\theta^B)$$ and then I apply the law of total probability
$$ P(w_i~|~\theta,\theta^A,\theta^B) = P(w_i~|~z_i=A,\theta,\theta^A,\theta^B)\cdot P(z_i=A~|~\theta,\theta^A,\theta^B) + P(w_i~|~z_i=B,\theta,\theta^A,\theta^B)\cdot P(z_i=B~|~\theta,\theta^A,\theta^B)$$
These terms can now be directly computed as
$$P(w_i~|~\theta,\theta^A,\theta^B)= \left( \prod_{j=1}^6 \theta_j^{A[w_i=j]}\right)\theta + \left( \prod_{j=1}^6 \theta_j^{B[w_i=j]}\right)(1-\theta)$$
(where $[x=i]$ is the Iverson Bracket)


*

*Now, in order to compute $P(\theta,\theta^A,\theta^B)$, I assume three independent priors, so $P(\theta,\theta^A,\theta^B) = P(\theta)\cdot P(\theta^A)\cdot P(\theta^B) $. For the coin I'll use a uniform Beta distribution and for the dice a uniform Dirichlet distribution (one for each).

*Finally, I could use the law of total probability again to compute $P(\mathbf{w})$, but I believe this to be intractable. Instead, I'm trying to understand how I can apply Gibbs sampling to this problem and estimate the bias parameters without computing $P(\mathbf{w})$.

If anything needs clarification, please say so. Any help would be appreciated, thanks in advance.
 A: 
How can I estimate the bias of the coin and the bias of each die using only the data he gives me?

You cannot.
Let the chances of the six outcomes of the first die be $p_1, p_2, \ldots, p_6$ and those of the second die be $q_1, q_2, \ldots, q_6$.  Let the chance of selecting the first die be $r$.  Then the expected frequency of observing the outcome $X$ for  $X=i$, $i=1, 2, \ldots, 6$, in your experiment is
$$\Pr(X=i) = r p_i + (1-r) q_i = t_i.$$
You can estimate the $t_i$ from the data, but that is all, because the $t_i$ completely determine all properties of the random variable $X$.  In particular, except in unusual circumstances (such as having at least one of $p_i$ and $q_i$ equal zero), you cannot even estimate $r$.
A: This problem is a typical latent variable problem, where you have unobserved data (the outcome of the coin). In this cases the parameters are usually estimated through the EM algorithm, which can deals with latent variables. Once the parameters are estimated you can evaluate the bias of the two distributions.
This paper will probably help you, you will just need to change the form of the likelihood as he is using two other coins instead of two dices.
http://www.cs.columbia.edu/~mcollins/6864/slides/em1.4up.pdf
