# Confusion related to likelihood

I was reading this paper http://stat.wharton.upenn.edu/~lzhao/papers/MyPublication/MultiExpert_ICML_2009.pdf

However, I have some confusion related to maximum likelihood derivation in this paper

We have the parameters of the model

$\theta = \{w,\alpha,\beta\}$

The maximum likelihood is given by

$P(D|\theta) = \prod_{i=1}^{i=N}P(y_i^{1},...y_i^{R}|x_i,\theta)$

where $y_i^1,...y_i^R$ are the noisy labels predicted by my R predictors.

Then the paper says conditioning on the true label, $y_i$ and also using the assumption that $\alpha^j$ and $\beta^j$ do not depend on the instance $x_i$ the likelihood can be written as

$P(D|\theta) = \prod_{i=1}^{i=N}P[y_i^1...y_i^R|y_i=1,\alpha].P[y_i=1,x_i,w] + P[y_i^1...y_i^R|y_i=0,\beta].P[y_i=0,x_i,w]$

I didn't get how this equation was derived. Any insights will be helpful.

The likelihood (not the maximum likelihood as you've written) is given by $$P\left[\mathcal{D}\mid\theta\right]=\prod_{i=1}^{N}P\left[y_i^1,\ldots,y_i^R\mid x_i,\theta\right]$$ This can be rewritten as the sum of the probabilities of two disjoint events.

$$P\left[\mathcal{D}\mid\theta\right]=\prod_{i=1}^{N}\left(P\left[y_i^1,\ldots,y_i^R,(y=0)\mid x_i,\theta\right]+P\left[y_i^1,\ldots,y_i^R,(y=1)\mid x_i,\theta\right]\right)$$

Using conditional probability we get that

\begin{eqnarray*} P\left[\mathcal{D}\mid\theta\right]&=&\prod_{i=1}^{N}\left(P\left[y_i^1,\ldots,y_i^R\mid x_i,\theta,(y=0)\right]P\left[y=0\mid x_i,\theta\right]+\right.\\ & & \left. P\left[y_i^1,\ldots,y_i^R\mid x_i,\theta,(y=1)\right]P\left[y=1\mid x_i,\theta\right]\right) \end{eqnarray*}

Once we condition on the event that $y$ is a certain value, each of the $y_i^j$ becomes a Bernoulli random variable with parameter either $\alpha^j$ (if $y=1$) or $\beta^j$ (if $y=0$); the $x_i$ and other parameters from $\theta$ do not affect the likelihood. This gives us that

\begin{eqnarray*} P\left[\mathcal{D}\mid\theta\right]&=&\prod_{i=1}^{N}\left(P\left[y_i^1,\ldots,y_i^R\mid \beta,(y=0)\right]P\left[y=0\mid x_i,\theta\right]+\right.\\ & & \left. P\left[y_i^1,\ldots,y_i^R\mid \alpha,(y=1)\right]P\left[y=1\mid x_i,\theta\right]\right) \end{eqnarray*}

Now we are also assuming that the probability that $y$ takes on a certain value depends only on $x$ and $w$, and no other parameters from $\theta$.

\begin{eqnarray*} P\left[\mathcal{D}\mid\theta\right]&=&\prod_{i=1}^{N}\left(P\left[y_i^1,\ldots,y_i^R\mid \beta,(y=0)\right]P\left[y=0\mid x_i,w\right]+\right.\\ & & \left. P\left[y_i^1,\ldots,y_i^R\mid \alpha,(y=1)\right]P\left[y=1\mid x_i,w\right]\right) \end{eqnarray*}

The important equations from the paper are $(3)-(5)$.