Maximum likelihood solution in probabilistic generative models

Question

I am reading Bishop's "Machine Learning and Pattern Recognition" (https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf) and I have the same question as in Maximum likelihood solution in classification problem, but unfortunately that answer is not clear to me.

We are considering the maximum likelihood solution at p. 200 in the context of probabilistic generative models for linear classification.

We are in the case of $K = 2$ classes, i.e., $\mathcal{C}_1$ and $\mathcal{C}_2$. We consider that each class has a Gaussian class-conditional density with shared covariance matrix; given a training set $\{\mathbf{x}_n, \mathbf{t}_n\}$, the likelihood function is given by $$p(\mathbf{t \mid \pi, \mathbf{\mu}_1, \mathbf{\mu}_2, \mathbf{\Sigma}}) = \prod_{n = 1}^N{[\pi\mathcal{N}(\mathbf{x}_n\mid \mathbf{\mu}_1, \mathbf{\Sigma})]^{\mathbf{t}_n}[(1 - \pi)\mathcal{N}(\mathbf{x}_n\mid \mathbf{\mu}_2, \mathbf{\Sigma}})]^{1 - \mathbf{t}_n},$$ but I do not understand why. It seems to me that the likelihood functions is the product of the joint distributions $p(\mathbf{x}_n, \mathcal{C}_1)$ and $p(\mathbf{x}_n, \mathcal{C}_1)$, instead of the product of the conditional distributions $p(\mathcal{C}_1 \mid \mathbf{x}_n)$ and $p(\mathcal{C}_2 \mid \mathbf{x}_n)$, as it should be in my opinion. Maybe am I missing something? Am I considering the likelihood function wrongly?

Thank you very much!

EDIT. As Demetri explained, there is a typo in the textbook and it should be $p(\mathbf{t}, \mathbf{X} \mid \pi, \mathbf{\mu}_1, \mathbf{\mu}_2, \mathbf{\Sigma})$ instead of $p(\mathbf{t} \mid \pi, \mathbf{\mu}_1, \mathbf{\mu}_2, \mathbf{\Sigma})$, so it makes sense now that the likelihood function is the product of the joint distributions $p(\mathbf{x}_n, \mathcal{C}_1)$ and $p(\mathbf{x}_n, \mathcal{C}_1)$, depending of the value of the output target. Anyway, since we have $$p(\mathcal{C}_k \mid \mathbf{x}) = \frac{p(\mathbf{x} \mid \mathcal{C}_k) p(\mathcal{C}_k)}{p(\mathbf{x})},$$ why do they call likelihood function the term $p(\mathbf{t}, \mathbf{X} \mid \pi, \mathbf{\mu}_1, \mathbf{\mu}_2, \mathbf{\Sigma})$, which is the product of the joint distributions $p(\mathbf{x} \mid \mathcal{C}_k) p(\mathcal{C}_k)$ and not of the likelihood functions $p(\mathbf{x} \mid \mathcal{C}_k)$? Is there any reason for this or is it another typo?

I hope that my doubt is clear, thank you!

Answer 1

Well, $P(\mathcal{C}_1 \vert \mathbf{x})$ is actually the thing we are trying to estimate from our data. We don't have access to $P(\mathcal{C}_1 \vert \mathbf{x}_n)$ for any particular $n$, all we know is that this observation either did or did not belong to $\mathcal{C_1}$. So we couldn't even take the product of $P(\mathcal{C}_1 \vert \mathbf{x})$ even if we wanted to.

That the likelihood is the product of the conditionals is just the definition of the likelihood. In most other texts, the likelihood is written as

$$ \prod_i f(x_i \vert \theta) $$

Here, $f$ is the density of $x$ and $\theta$ is a parameter for the density. The expression written above is actually an equivalent formulation since we are assuming that $\mathbf{x}$ is normal and the classes share a mean and covariance matrix.

By the way, there is a typo in the printed version of this book for anyone reading this answer.