# When is OLS=MLE and when is it not? [duplicate]

In linear regression, OLS and MLE are equal to each other. Is this just a coincidence or is there a reason? Is there some way of knowing when/why they will be equal to each other and when they will not?

• MLE for what likelihood?
– Dave
Commented Oct 29, 2023 at 19:07
• I recommend that you ponder Dave's question carefully. Then think about properties of the likelihood function you come up with, and how that relates to a loss you might minimize. Commented Oct 29, 2023 at 22:13

Suppose the model is $$Y_i = \alpha+\beta x_i + \varepsilon_i$$ where $$\alpha,\beta$$ are non-random and not observable, and $$x_i$$ is treated as non-random, but is observable, and $$\varepsilon_i, i=1,\ldots,n$$ are i.i.d. $$\operatorname N(0,\sigma^2),$$ and $$\sigma^2$$ is not observable and $$Y_i, i=1,\ldots,n$$ are observable.

Then the MLE for $$\alpha,\beta$$ is the least-squares estimator.

The harder question is whether the converse holds: If the MLE coincides with the ordinary-least-squares estimators, is the distribution of errors necessarily what is described above?

Here I will continue to assume that the errors are i.i.d., but there is a question of whether independence or identical distribution or both might be somehow deduced from the fact that OLS coincides with MLE.

\begin{align} \log L(\alpha,\beta) = {} & \log \prod_{i=1}^n f(y_i-(\alpha+ \beta x_i)) \\[8pt] = & {} \sum_{i=1}^n \log f(y_i-(\alpha+\beta x_i)) \end{align}

So we have

\begin{align} & \operatorname*{argmin}_{\alpha,\beta} \sum_{i=1}^n (y_i - (\alpha+\beta x_i))^2 \\[8pt] = {} & \operatorname*{argmax}_{\alpha,\beta} \sum_{i=1}^n \log f(y_i-(\alpha+\beta x_i)). \end{align}

This should hold for all values of $$x_i, y_i,i=1,\ldots,n.$$

If one can show that it follows that $$\log f(w) = \Big(\text{negative constant} \times w^2 \Big),$$ then there you have a Gaussian distribution.