# Why is Maximum Likelihood used for ARIMA instead of least squares?

Having decided the orders (p,q,d) of an ARIMA model - why do we use Maximum Likelihood Estimation instead of least squares to determine the coefficients?

An ARIMA model is after all very similar to a multivariable regression.

• I think this has been asked before, so I am only answering this in the comment: when $q>0$, OLS cannot be applied since the necessary ingredient are missing; you do not have the design matrix $X$ since you do not observe the lagged errors. If you had observable lagged errors, you could surely use OLS, but you don't. Meanwhile, for the case $q=0$, maximum likelihood and OLS are almost the same. – Richard Hardy Jan 16 '18 at 13:54
• Possible duplicate of What is/are the "mechanical" difference between multiple linear regression with lags and time series?. The question is different, but the answer answers it, so not really sure if it should be a duplicate or not. – Richard Hardy Jan 16 '18 at 14:00
• @RichardHardy Interesting point, personally I have voted to leave this one open. In fact, if anything, some of the answers there really would be a better fit here! (Within its scope, this seems to be a "better" question - very clear and to the point.) Though I don't think the other question could be closed as a duplicate of this one either, because its scope seems to spread wider. – Silverfish Jan 16 '18 at 14:55
• @Silverfish, you are probably right. Should the relevant part of the answer be just copied here? – Richard Hardy Jan 16 '18 at 15:11
• @RichardHardy That seems perfectly appropriate to me! – Silverfish Jan 16 '18 at 19:06

OLS only works when all regressors are observable, because it employs the design matrix $X$ in estimating the model coefficients ($\beta=(X'X)^{-1}X'y$). Meanwhile, the ARIMA model specification contains unobservable variables in the moving average part (the lagged errors); hence, the OLS estimator is infeasible when the moving-average order $q>0$. On the other hand, an AR model can be estimated with OLS, and this is in fact quite a common approach. For normally distributed errors, it is also very close to the Maximum Likelihood (ML) solution. (There are subtle differences on how to treat the initial values in the ML estimation, but other than that it coincides with OLS for AR models.)
• Interesting. It is quite obvious that a direct OLS estimation of MA(q) and ARMA(p,q) with $q>0$ models is impossible due to the unobservable regressors. I would therefore not assume that Eviews uses OLS even when $q>0$ for MA(q). The closest one can get to OLS estimation of these models is a two-stage procedure where in stage 1 an AR(r) model is fit with a relatively large $r$, and then the residuals of this model are used as proxies for the unobservable lagged error terms in stage 2 where OLS is used. – Richard Hardy Jan 22 '18 at 20:07