# Conditional Maximum Likelihood Estimation for ARMA(p,q)

I'm making a study on STARIMA (Space-Time Autoregressive Integrated Moving Average) Time Series. And one problem I have is the parameter estimation since there is no software yet that has a STARIMA function, so far from what I have searched. Now, I want to compute it manually using estimation procedure. And I prefer Conditional Maximum Likelihood Estimation as it is used in STARIMA. I want to start with the univariate time series, the ARMA(p,q). I wish to estimate first the parameters of ARMA(p,q), then I'll just apply the procedure in STARMA.

According to William W.S. Wei author of Time Series Analaysis: Univariate and Multivariate Methods that (Original text from his book):

For the general stationary ARMA(p,q) model$$Z_t^*=\phi_1Z_{t-1}^*+\cdots+\phi_pZ_{t-p}^*+a_t-\theta_1a_{t-1}-\cdots-\theta_qa_{t-q}\quad(7.2.1)$$where $Z_t^*=Z-\mu$ and $\{a_t\}$ are i.i.d. $N(0,\sigma_a^2)$ white noise, the joint probability density of $\mathbf{a}=(a_1,a_2,\cdots,a_n)'$ is given by$$P(\mathbf{a}|\phi,\mu,\theta,\sigma_a^2)=(2\pi\sigma_a^2)^{-n/2}exp\left[-\frac{1}{2\sigma_a^2}\sum_{t=1}^na_t^2\right].\quad(7.2.2)$$Rewriting $(7.2.1)$ as$$a_t=\theta_1a_{t-1}+\cdots+\theta_qa_{t-q}+Z_t^*-\phi Z_{t-1}^*-\cdots-\phi_pZ_{t-p}^*\quad(7.2.3)$$we can write down the likelihood function of the parameters $(\phi,\mu,\theta,\sigma_a^2)$. Let $\mathbf{Z}=(Z_1,Z_2,\cdots,Z_n)'$ and assume the initial conditions $\mathbf{Z_*}=(Z_{1-p},\cdots,Z_{-1},Z_0)'$ and $\mathbf{a_*}=(a_{1-q},\cdots,a_{-1},a_0)$. The conditional log-likelihood function$$\ln L_*(\phi,\mu,\theta,\sigma_a^2)=-\frac{n}{2}\ln 2\pi\sigma_a^2-\frac{S_*(\phi,\mu,\theta)}{2\sigma_a^2}\quad(7.2.4)$$where$$S_*(\phi,\mu,\theta)=\sum_{t=1}^na_t^2(\phi,\mu,\theta|\mathbf{Z_*},\mathbf{a_*},\mathbf{Z})\quad(7.2.5)$$is the conditional sum of squares function. The quantities of $\widehat{\phi},\widehat{\mu}$ and $\widehat{\theta}$, which maximize Equation $(7.2.4)$, are called the conditional maximum likelihood estimators.

Now, I don't have idea how to find the quantities $\widehat{\phi},\widehat{\mu}$ and $\widehat{\theta}$ that maximizes $(7.2.4)$. Is this the same with the usual Maximum Likelihood Estimation like for parameters of say Geometric Distribution? If so, how will I obtain the formula of each parameters ($\widehat{\phi},\widehat{\mu}$ and $\widehat{\theta}$)? I know its tedious to perform MLE, but just give me an idea how to set up the likelihood function, and I'll just perform the usual MLE for parameters of ARMA(p,q). Well, if you could give me other source for parameter estimation of STARIMA then that would be great.

• Sorry for late response, there was a power interruption in our place. Thanks for your answer Michael, I never encountered problems yet with conditional maximum likelihood computation, what I know is the general MLE only which I always use in estimating the parameters. Any idea how to set up the conditional maximum likelihood function for the three quantities ($\widehat{\phi} ,\widehat{\mu}$ , and $\widehat{\theta}$)? – Al-Ahmadgaid Asaad Sep 3 '12 at 8:44
• If I'm going to simplify equation $(7.2.4)$ using the Maximum Likelihood Estimation Computation, I will be stacked at taking the first derivative of equation $(7.2.5)$ with respect to say $\phi$ parameter. – Al-Ahmadgaid Asaad Sep 3 '12 at 8:53
• Equation 7.2.4 is the conditional log likelihood function. The terms maximum and estimator have nothing directly to do with the equation. What is the difficulty taking partial derivatives of the function with respect to a$_t$? – Michael R. Chernick Sep 3 '12 at 11:32
• I'm sorry, Yes you're right Equation 7.2.4 is the conditional log likelihood function. But, I'm having difficulty of taking the partial derivatives. What I know on taking the estimator of say $\phi$ from equation 7.2.4 is that I need to take the partial derivatives of equation 7.2.4 with respect to $\phi$, but I don't have idea on taking the partial derivative of that especially on the term $S_*(\phi , \mu , \theta)$. Thanks for replying. – Al-Ahmadgaid Asaad Sep 3 '12 at 11:55
• The phi parameters are linear in the Z$_t$* and the theta parameters in the a$_t$s. The partial derviatives with respect to the phis and the thetas are just the Z$_t$s and the a$_t$s respectively. – Michael R. Chernick Sep 3 '12 at 12:01