# Deriving conditional maximum likelihood

When maximum likelihood is used to fit a parameter of a conditional distribution, I often see the argument go like this. We have our data set of experienced tuples $$\mathcal{D} = \{(x_1,y_n), \cdots , (x_n, y_n)\}$$. When we measure the conditional likelihood $$L(\theta) = P(y_1, \cdots, y_n \mid x_1, \cdots, x_n) \;,$$ we can decompose it as $$L(\theta) = L^{*}(\theta) = \prod_{i=1}^n P(y_i \mid x_i) \; ,$$ based off some assumptions. What are those assumptions, and can someone provide the full derivation of the likelihood from $$L$$ to $$L^*$$? For example, is $$(x_i, y_i)$$ assumed to be i.i.d for $$i = \{1, \cdots ,n\}$$?

I think I have a solution. We assume that $$(x_i, y_i)$$ is i.i.d. for $$i = \{1, \cdots, n\}$$. Then the joint likelihood $$L(\theta) = P((x_1, y_1), \cdots, (x_n, y_n)),$$ becomes $$L(\theta) = P\Big(x_1, y_1\Big)P\Big((x_2, y_2), \cdots, (x_n, y_n) \mid (x_1, y_1)\Big) = \cdots = \prod_{i = 1}^nP(x_i, y_i)$$ where I apply the i.i.d. assumption $$n$$ times. Then, when fitting the conditional distribution, we use Bayes rule $$L(\theta) = \prod_{i = 1}^nP(x_i)P(y_i \mid x_i),$$ and ignore the marginal distribution $$p(x_i)$$ when we are maximizing as we are only parameterizing $$p(y_i \mid x_i)$$.
My question is: is the i.i.d. assumption for $$(x_i, y_i)$$ where $$i = \{1, \cdots, n\}$$ typically the assumption used when doing conditional maximum likelihood?
Note that instead of saying "we are only parameterizing $$p(y_i \mid x_i)$$", you could be more explicit by including $$\theta$$ at all stages. You could even go one step further by replacing $$p(x,y)=p(y|x)p(x)$$ with $$p(x,y|\theta_X,\theta_Y) = p(y|x, \theta_X,\theta_Y) p(x|\theta_X,\theta_Y) = p(y|x, \theta_Y) p(x|\theta_X).$$ The second equality is based on the assumption that the distribution of $$Y$$ depends on parameter $$\theta_Y$$ (but not $$\theta_X$$), and similar for $$X$$. I believe this is an important assumption we can't avoid if we want the optimisation problem $$\max_{\theta_{X}}\prod_{i}p(x_{i}|\theta_{X})$$ to be equivalent to $$\max_{\theta_{X}}\prod_{i}p(x_{i},y_{i}|\theta_{X},\theta_{Y})=\max_{\theta_{X}}\prod_{i}p(y_{i}|x_{i},\theta_{X},\theta_{Y})p(x_{i}|\theta_{X},\theta_{Y})$$ and necessary (that is, to be a sub-problem) for $$\max_{\theta_{X},\theta_{Y}}\prod_{i}p(x_{i},y_{i}|\theta_{X},\theta_{Y})=\max_{\theta_{X},\theta_{Y}}\prod_{i}p(y_{i}|x_{i},\theta_{X},\theta_{Y})p(x_{i}|\theta_{X},\theta_{Y}).$$
I find it helpful to use the graphical models formalism. In terms of graphical models, our independence assumptions include $$p(y|x, \theta_X,\theta_Y) = p(y|x, \theta_Y)$$ and are described by the following diagram: $$\begin{array}{ccc} \theta_{X} & & \theta_{Y}\\ \downarrow & & \downarrow\\ \left(X\right) & \to & \left(Y\right) \end{array}$$ In contrast, the following diagram permits that $$p(y|x, \theta_X,\theta_Y) \neq p(y|x, \theta_Y)$$: $$\begin{array}{ccc} \theta_{X} & & \theta_{Y}\\ \downarrow & \searrow & \downarrow\\ \left(X\right) & \to & \left(Y\right) \end{array}$$ Here $$\left(X\right),\left(Y\right)$$ are random variables we sample, and $$\theta_{X}, \theta_{Y}$$ are parameters constant throughout our observation or experiment. That is, $$(X_i,Y_i)$$ are i.i.d. and parameterised by $$\theta_{X}, \theta_{Y}$$.