Maximum Likelihood Formulation for Linear Regression I have seen the following for maximum likelihood estimation (MLE) for linear regression in multiple sources, e.g. here:
$$
\mathcal{D} \equiv \{(x_1, y_1), ..., (x_n, y_n)\}
$$
I do not understand how exactly we derive this:
$$
p(\mathcal{D} | \theta) = \prod_{i=1}^n p(y_i | x_i, \theta)
$$
I understand that we can write the product due to the assumtion of independent $y_i$. However, I do not understand why $x_i$ is suddenly on the right side. Shouldn't it be:
$$
p(\mathcal{D} | \theta) = \prod_{i=1}^n p(y_i , x_i | \theta)
$$
 A: In ordinary least squares regression the goal is to model the condition expectation;
$$
E[y_i|x_i] = x_i'\beta
$$
$y_i$ and $x_i$ are referred to as the dependent and independent variables respectively because we are literally conditioning  $y_i$ on $x_i$.  
Ordinary least squares is equivalent to maximum likelihood where we assume;
$$
y_i|x_i \stackrel{iid}{\sim} N(x_i'\beta,\sigma^2)
$$
In this instance the $x_i$ are taken as fixed values (we are not calling $x_i$ a random variable and giving it a probability distribution) meaning that the "data", $\mathcal{D}$, is just the set of $y_i$'s
$$\mathcal{D} \equiv \{y_1,..,y_n\}$$
So writing 
$$
p(\mathcal{D} | \theta) = \prod_{i=1}^n p(y_i | x_i, \theta)
$$
where $\theta \equiv \{\beta,\sigma\}$ is actually correct.
The likelihood $p(\mathcal{D} | \theta) = \prod_{i=1}^n p(y_i , x_i | \theta)=\prod_{i=1}^n p_y(y_i  | x_i, \theta)p_x(x_i|\theta)$ ,on the other hand, treats the $x_i$ as random variables which, although applicable in some settings, is not linear regression in the traditional sense.
