In normal linear regression, defining $Y_i = X_i\beta+\epsilon_i,$ with $\epsilon_i$ c.i.i.d. $\mathcal{N}(0,\sigma^2)$, is equivalent to say $Y|X \sim \mathcal{N}_n(X\beta,\sigma^2I_n)$. The second seems a better choice for generalization, for instance in GLM's, where the distribution of $Y|X$ usually belongs to the exponential family.

In practice, if there are no "physical constraints" in choosing the statistical model, how does one model the stochastic part of a GLM? I used to think that you could do it by looking at the histogram of $Y$, but this question What if residuals are normally distributed, but y is not? made me realize I was completely wrong!