# Maximum likelihood estimator - conditioning input on model parameters

Assume we have a dataset $$D=\{(x_i, y_i)\}$$ drawn from the joint distribution $$(x_i, y_i) \sim P (x=x_i, y=y_i)$$. We want to make predictions based on the dataset and for that we use a parametric model with parameter $$\theta$$ so that the prediction for a new $$x^*$$ is $$P(y^*|x^*, \theta)$$.

To estimate $$\theta$$, we maximize the likelihood $$P(D|\theta)$$ with respect to $$\theta$$, i.e. we find $$\theta$$ that maximizes the likelihood of observing the dataset $$D$$. To keep things simple, let us assume there is a single data point in the set, $$D=\{(x_1, y_1 \}$$. Then the likelihood is $$P(D|\theta) = P((x_1, y_1)|\theta)$$, i.e. the joint probability distribution of $$(x_1, y_1)$$ conditioned on $$\theta$$. We can expand the joint distribution as $$P((x_1, y_1)|\theta)=P(y_1|x_1,\theta)P(x_1|\theta)$$. However, in the usual definition of likelihood, only the first term appears. My question is, what happens to the second term $$P(x_1|\theta)$$? Is that somehow independent of $$\theta$$ and thus irrelevant to the estimator? If so, how can I see that?

• You wish to predict $y$ based on an observed $x$, so the only thing that matters is $p(y|x, \theta)$. $p(x|\cdot)$ doesn't matter because you will have observed $x$ at the time you are making your prediction, so it is fixed. Your likelihood function needs to reflect that. Jan 3 at 2:17