# Where do the error terms come from in ARMA?

I understand ARMA is a linear combination of lagged data points and lagged errors, but I am unclear on its implementation once parameters have been identified. Now suppose I have an ARMA model and some data. Where do I get these error terms, especially for the first term.

They are unobservable, just like errors in a regression model. You can estimate them, again just like in a regression model. Estimation of ARMA models is done via maximum likelihood, frequently via state-space representation and Kalman filtering. A good description of ARMA estimation is available Hamilton "Time Series Analysis" and other time series textbooks (and probably some threads on Cross Validated; search for estimation and ARIMA).
• Thank you! I have been doing some reading since I posted the question and it seems for an ARMA (1,1) process I would arbitrarily start with $\epsilon_0 = 0$ and recursively compute the sequence of $\epsilon_i$ through the ARMA equation, but rewritten as: $\epsilon_t = X_t - \mu - \theta X_{t-1} - \phi \epsilon_{t-1}$. With some assumed distribution, I take the maximum likelihood of these $\epsilon_t$ using the distribution's PDF. With $\mu$ being the mean of the data, I use this process to find $\theta$ and $\phi$. Am I on the right track? Commented Apr 27, 2021 at 6:23
• @CBBAM, yes, I think so. There are two caveats, though. First, this is conditional likelihood. In full likelihood, you would optimize for $\epsilon_0$, too. Second, naive optimization of the likelihood may be highly inefficient, take long time and fail to converge. (I think I tried it some years ago, and it did not go well.) Just look at the explicit expression of the likelihood in terms of the observed data and the parameters; the likelihood function is highly nonlinear w.r.t. the parameters, and that gets worse with sample size, if I remember correctly. Commented Apr 27, 2021 at 16:51
• Thank you, how big of an affect does not optimizing $\epsilon_0$ have? Commented Apr 27, 2021 at 17:00