Does least squares regression imply normality of errors?

Question

For the linear model

$$y_i=\beta_0 +\sum_{k=1}^{n}\beta_k x_{ik} + \epsilon_i$$

the parameter estimates are the same for the maximum likelihood method and the least square method (minimizing $\sum_{i=1}^n\epsilon_i^2$) if the errors are assumed iid normal. So I am thinking that if we use least squares, we are perhaps implicitly assigning normality to the errors. Is this correct - can it be shown that the least square method does this (without appealing to the maximum likelihood solution)?

Answer 1

There are simple examples where the MLE minimizes the SSE but the distribution is not normal. For starters, notice that the sample mean is the least-squares estimator of the location of a univariate sample $x_1,x_2,\ldots,x_n$, as it is the $\theta$ that minimizes $\sum(x_i-\theta)^2$.

Now consider the exponential distribution, $f(x) = \frac{1}{\theta} e^{-x/\theta}$, $x>0$; or the Poisson distribution with pmf $p(x) = e^{-\theta}\theta^x/x!$, $x=0,1,2,\ldots$. In both cases, given random samples from these distributions, the MLE of $\theta$ is the sample mean. So these are examples where the distribution is non-normal but the MLE is the least-squares estimator.

Answer 2

When you are doing the OLS estimation, you are just using a criteria for your estimation ( that is "minimize the sum of the squared residuals"), and so a particular method. You do not need anything (not normally distributed residuals nor anything) to do it and doing it does not imply anything. Whether this method is the best (for some definition of the best) depends on the assumptions you make.

Traditionally, in econometrics, it said to be the best estimator among the other ones if some assumptions are satisfied.

• Without assuming normality of the residuals (but with other assumptions), OLS is the BLUE (Best Linear Unbiased Estimator).
• If the residuals are normally distributed, OLS then become BUE (Best Unbiased Estimator)

Check here for the precise assumptions