EM Derivation for Dawid-Skene Model I am trying to derive the EM update equations for the Dawid-Skene model. Following the notation in Bayesian Classifier Combination by Kim and Ghahramani, $i$ is the index of the data point, $t_i$ is the true label generated by a multinomial distribution with parameters $\mathbf{p}$, and  $c_i^{(k)}$ is the output of the $k$th classifier which is also generated by a multinomial distribution with parameters $\pi_j^{(k)}$ where $j \in \lbrace 1, ..., J \rbrace$ is the number of classes.
\begin{eqnarray}
p(t_i=j|\mathbf{p})=p_j\\
p(c_i^{(k)}|t_i=j, \pi)=\pi^{(k)}_{j, c_i^{(k)}}\\
p(\mathbf{c}_i|t_i=j, \pi)=\prod_{k=1}^K \pi^{(k)}_{j, c_i^{(k)}}\\
p(\mathbf{c}_i, t_i=j|\mathbf{p}, \pi)=p_j\prod_{k=1}^K \pi^{(k)}_{j, c_i^{(k)}}\\
p(\mathbf{c}_i, t_i|\mathbf{p}, \pi)=p_{t_i}\prod_{k=1}^K \pi^{(k)}_{t_i, c_i^{(k)}}\\
p(\mathbf{c}, \mathbf{t}|\mathbf{p}, \pi)=\prod_{i=1}^I \left( p_{t_i}\prod_{k=1}^K \pi^{(k)}_{t_i, c_i^{(k)}} \right)
\end{eqnarray}
The paper says "By considering $t_i$ as hidden variables, we can apply the EM algorithm to find ML estimates for $\mathbf{p}$ and $\mathbf{\pi}$". So should I write $Q(\theta, \theta^{(t-1)})$ as $E_{\mathbf{t}|\mathbf{c}, \mathbf{p}, \pi}[\log p(\mathbf{c}, \mathbf{t}|\mathbf{p}, \pi)|\mathbf{c},\mathbf{p}^{(t-1)},\pi^{(t-1)}]$ or $E_{t_i|\mathbf{c}, \mathbf{p}, \pi}[\log p(\mathbf{c}, \mathbf{t}|\mathbf{p}, \pi)|\mathbf{c},\mathbf{p}^{(t-1)},\pi^{(t-1)}]$ or something else?
Moreover, here I need something like $p(t_i|\mathbf{c}_i, \mathbf{p})$, but I don't have it, so how should I proceed?
What would be the correct notation here for expectations, subscript ($E_{t_i|\mathbf{c},\mathbf{p},\pi}[\log p(\mathbf{c},\mathbf{t}|\mathbf{p},\pi)]$), condition ($E[\log p(\mathbf{c},\mathbf{t}|\mathbf{p},\pi)|\mathbf{c},\mathbf{p},\pi]$), or both ($E_{t_i|\mathbf{c},\mathbf{p},\pi}[\log p(\mathbf{c},\mathbf{t}|\mathbf{p},\pi)|\mathbf{c},\mathbf{p},\pi]$)?
In the M step with respect to which terms should I take the derivatives?
And in this paper, on page 2, expectation is defined as $E[h(Y)|X=x]=\int_y h(y)f_{Y|X}(y|x)dy$ in the footnote, but in equation 2 for EM it is written as $E[\log p(X,Y|\theta)|X,\theta^{(i-1)}]=\int_{\mathbf{y}\in\mathbf{Y}}\log p(X,\mathbf{y}|\theta)f(\mathbf{y}|X,\theta^{(i-1)})d\mathbf{y}$. $h(Y)$ is a function of $Y$ and $\log p(X,Y|\theta)$ is a function of $X$ and $Y$, so why don't we write this expectation with respect to $f(\mathbf{x},\mathbf{y}|X,\theta^{(i-1)})$?
 A: First consider the case that you have no gold answers. i.e. no $t_i$ known.
For simplicity, consider only one sample $i$ here.
Then the negative likelihood function (loss function) is $$\mathcal{L}(\mathbf{c}_i; \mathbf{p}, \pi) = -\log p(\mathbf{c}_i | \mathbf{p}, \pi)$$, where $\mathbf{p}$ and $\pi$ are the parameters of the likelihood function. No matter what optimization method you use, SGD or EM, the MLE objective is this one.
Expanding the loss function gives
\begin{equation}
\begin{split}
\mathcal{L}(\mathbf{c}_i; \mathbf{p}, \pi) &= -\log p(\mathbf{c}_i | \mathbf{p}, \pi) \\
&= -\log \sum_{j=1}^J p(\mathbf{c}_i, j| \mathbf{p}, \pi) \\
&= -\log \sum_{j=1}^J p_{j}\prod_{k=1}^K \pi^{(k)}_{j, c_i^{(k)}}\\
\end{split}
\end{equation}
Now you can see this is a mixture of multinomial model. Just like mixture of gaussians, let $\mathbf{t}_i$ be the $J$ item vector denoting the posterior estimation of problem $i$, the E step should be
$$ \forall j, \mathbf{t}_i[j] := p(t_i = j | \mathbf{c}_i, \mathbf{p}, \pi) = \frac{p_{j}\prod_{k=1}^K \pi^{(k)}_{j, c_i^{(k)}}}{\sum_{j'=1}^J p_{j'}\prod_{k=1}^K \pi^{(k)}_{j', c_i^{(k)}}}$$
and M step should be 
$$ \mathbf{p} \leftarrow \frac{1}{I} \sum_{i=1}^I \mathbf{t}_i $$ and 
$$ \forall j, k, \pi^{(k)}_{j, \cdot} \leftarrow \sum_{i=1}^I \mathbf{t}_i[j] \pi_{j, \cdot}^{(k)} $$
IMHO, in the deep learning era, for most tasks, gradient descent can replace EM entirely. There will not be much performance degrade. It saves the time to derive and debug the formula.
