I understand ARMA is a linear combination of lagged data points and lagged errors, but I am unclear on its implementation once parameters have been identified. Now suppose I have an ARMA model and some data. Where do I get these error terms, especially for the first term.


1 Answer 1


They are unobservable, just like errors in a regression model. You can estimate them, again just like in a regression model. Estimation of ARMA models is done via maximum likelihood, frequently via state-space representation and Kalman filtering. A good description of ARMA estimation is available Hamilton "Time Series Analysis" and other time series textbooks (and probably some threads on Cross Validated; search for estimation and ARIMA).

  • $\begingroup$ A starting point could be "ARIMA estimation by hand. $\endgroup$ Commented Apr 27, 2021 at 6:05
  • $\begingroup$ Thank you! I have been doing some reading since I posted the question and it seems for an ARMA (1,1) process I would arbitrarily start with $\epsilon_0 = 0$ and recursively compute the sequence of $\epsilon_i$ through the ARMA equation, but rewritten as: $\epsilon_t = X_t - \mu - \theta X_{t-1} - \phi \epsilon_{t-1}$. With some assumed distribution, I take the maximum likelihood of these $\epsilon_t$ using the distribution's PDF. With $\mu$ being the mean of the data, I use this process to find $\theta$ and $\phi$. Am I on the right track? $\endgroup$
    – CBBAM
    Commented Apr 27, 2021 at 6:23
  • $\begingroup$ @CBBAM, yes, I think so. There are two caveats, though. First, this is conditional likelihood. In full likelihood, you would optimize for $\epsilon_0$, too. Second, naive optimization of the likelihood may be highly inefficient, take long time and fail to converge. (I think I tried it some years ago, and it did not go well.) Just look at the explicit expression of the likelihood in terms of the observed data and the parameters; the likelihood function is highly nonlinear w.r.t. the parameters, and that gets worse with sample size, if I remember correctly. $\endgroup$ Commented Apr 27, 2021 at 16:51
  • $\begingroup$ Thank you, how big of an affect does not optimizing $\epsilon_0$ have? $\endgroup$
    – CBBAM
    Commented Apr 27, 2021 at 17:00
  • $\begingroup$ @CBBAM, I presume it is small when the sample size is medium to large. If you get around writing code for ARMA or ARMA-GARCH, it would be cool to see it. Consider posting it by answering one of your questions such as this one. I would appreciate if you pinged me then. Thank you. $\endgroup$ Commented Apr 27, 2021 at 17:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.