I understand ARMA is a linear combination of lagged data points and lagged errors, but I am unclear on its implementation once parameters have been identified. Now suppose I have an ARMA model and some data. Where do I get these error terms, especially for the first term.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
6
They are unobservable, just like errors in a regression model. You can estimate them, again just like in a regression model. Estimation of ARMA models is done via maximum likelihood, frequently via state-space representation and Kalman filtering. A good description of ARMA estimation is available Hamilton "Time Series Analysis" and other time series textbooks (and probably some threads on Cross Validated; search for estimation and ARIMA).
answered Apr 27, 2021 at 6:02
-
1$\begingroup$ A starting point could be "ARIMA estimation by hand. $\endgroup$ Commented Apr 27, 2021 at 6:05
-
$\begingroup$ Thank you! I have been doing some reading since I posted the question and it seems for an ARMA (1,1) process I would arbitrarily start with $\epsilon_0 = 0$ and recursively compute the sequence of $\epsilon_i$ through the ARMA equation, but rewritten as: $\epsilon_t = X_t - \mu - \theta X_{t-1} - \phi \epsilon_{t-1}$. With some assumed distribution, I take the maximum likelihood of these $\epsilon_t$ using the distribution's PDF. With $\mu$ being the mean of the data, I use this process to find $\theta$ and $\phi$. Am I on the right track? $\endgroup$– CBBAMCommented Apr 27, 2021 at 6:23
-
$\begingroup$ @CBBAM, yes, I think so. There are two caveats, though. First, this is conditional likelihood. In full likelihood, you would optimize for $\epsilon_0$, too. Second, naive optimization of the likelihood may be highly inefficient, take long time and fail to converge. (I think I tried it some years ago, and it did not go well.) Just look at the explicit expression of the likelihood in terms of the observed data and the parameters; the likelihood function is highly nonlinear w.r.t. the parameters, and that gets worse with sample size, if I remember correctly. $\endgroup$ Commented Apr 27, 2021 at 16:51
-
$\begingroup$ Thank you, how big of an affect does not optimizing $\epsilon_0$ have? $\endgroup$– CBBAMCommented Apr 27, 2021 at 17:00
-
$\begingroup$ @CBBAM, I presume it is small when the sample size is medium to large. If you get around writing code for ARMA or ARMA-GARCH, it would be cool to see it. Consider posting it by answering one of your questions such as this one. I would appreciate if you pinged me then. Thank you. $\endgroup$ Commented Apr 27, 2021 at 17:00