So the motivation for using the squared loss in linear regression can be written as the following (I think):
Assume $\{(\mathbf{x}_i, y_i) \mid i = 1, \dots n\}$ are repeated independent samples from random variables $\mathbf{X}$ and $Y$ respectively. With $Y \mid \mathbf{X} \sim N(f(\mathbf{x};\mathbf{w}), \sigma^2)$.
We then can write that:
$$ p(y_1, \dots y_n \mid \mathbf{x}_1, \dots \mathbf{x}_n; \mathbf{w}, \sigma) = \frac{\prod_{i=1}^{n}{p(\mathbf{x}_i,y_i;\mathbf{w}, \sigma)}}{\prod_{i=1}^{n}{p(\mathbf{x}_i;\mathbf{w}, \sigma)}} \\ = \prod_{i=1}^{n}{p(y_i \mid \mathbf{x}_i;\mathbf{w}, \sigma)} $$
And therefore under the distributional assumption the log-likelihood is given by:
$$\begin{align} \log(p(y_1, \dots y_n \mid \mathbf{x}_1, \dots \mathbf{x}_n; \mathbf{w}, \sigma)) = C + \frac{\sum_{i=1}^{n}{(y_i - f(\mathbf{x}_i;\mathbf{w}))^2}}{2\sigma^2} \end{align}$$
And hence the MLE of $\mathbf{w}$ is given by solving the least squares problem.
Above we assumed $(\mathbf{x}_i, y_i) \perp (\mathbf{x}_j, y_j)$ for $i \neq j$.
But I think we can get the same with just assuming conditional independence i.e $y_i \mid \mathbf{x}_i \perp y_j \mid \mathbf{x}_j$ for $i \neq j$. But, I do not think that this notation is valid, is the correct notation $y_i \perp y_j \mid \mathbf{x}_j, \mathbf{x}_i$. But then I do not think this correct as that statement only allows us to write:
$$ p(y_i, y_j \mid \mathbf{x}_i, \mathbf{x}_j) = p(y_i \mid \mathbf{x}_i, \mathbf{x}_j) p(y_j \mid \mathbf{x}_i, \mathbf{x}_j) $$
But we really want:
$$ p(y_i, y_j \mid \mathbf{x}_i, \mathbf{x}_j) = p(y_i \mid \mathbf{x}_i) p(y_j \mid \mathbf{x}_j) $$
In summary, my question basically is, what is the correct way to write the conditional independent statements for linear regression?
