Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Assume that I'm going to estimate a linear regression where I assume $u\sim N(0,\sigma^2)$. What is the benefit of OLS against ML estimation? I know that we need to know a distribution of $u$ when we use ML Methods, but since I assume $u\sim N(0,\sigma^2)$ whether I use ML or OLS this point seems to be irrelevant. Thus the only advantage of OLS should be in the asymptotic features of the $\beta$ estimators. Or do we have other advantages of the OLS method?

share|improve this question
add comment

3 Answers

Using the usual notations, the log-likelihood of the ML method is

$l(\beta_0, \beta_1 ; y_1, \ldots, y_n) = \sum_{i=1}^n \left\{ -\frac{1}{2} \log (2\pi\sigma^2) - \frac{(y_{i} - (\beta_0 + \beta_1 x_{i}))^{2}}{2 \sigma^2} \right\}$.

It has to be maximised with respect to $\beta_0$ and $\beta_1$.

But, it is easy to see that this is equivalent to minimising

$\sum_{i=1}^{n} (y_{i} - (\beta_0 + \beta_1 x_{i}))^{2} $.

Hence, both ML and OLS lead to the same solution.

More details are provided in these nice lecture notes.

share|improve this answer
    
Thanks for your answer ocram. It's clear that both methods lead to the same solution. But the OLS should be more powerful since the estimators are more efficient than the ML estimators, no? I'm wondering about the differences and the advantages of both methods in the context when assuming $u\sim N(0,\sigma^2)$ and I'm interested in the features of the estimators. The values for $\beta$ are identic, but I've in mind that the asymptotic features of OLS estimators are preferable. roughly spoken: If ML and OLS give the same results and features of the estimators, why should we use OLS? –  MarkDollar Jul 21 '11 at 12:32
10  
The maximum likelihood estimator is OLS. As they are exactly the same, they will have the same asymptotic properties. –  Simon Byrne Jul 21 '11 at 13:18
    
+1; Thank you for stating that so clearly @SimonByrne –  rpierce Jul 25 '13 at 1:24
add comment

You are focusing on the wrong part of the concept in your question. The beauty of least squares is that it gives a nice easy answer regaurdless of the distribution, and if the true distribution happens to be normal, then it is the maiximum likelihood answer as well (I think this is the Gauss-Markov thereom). When you have a distribution other than the normal then ML and OLS will give different answers (but if the true distribution is close to normal then the answers will be similar).

share|improve this answer
add comment

the only difference for finite samples is, that the ML-estimator for the residual variance is biased. It does not account for the number of regressors used in the model.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.