ARIMA vs ARMA on the differenced series

Question

In R (2.15.2) I fitted once an ARIMA(3,1,3) on a time series and once an ARMA(3,3) on the once differenced timeseries. The fitted parameters differ, which I attributed to the fitting method in ARIMA.

Also, fitting an ARIMA(3,0,3) on the same data as ARMA(3,3) will not result in identical parameters, no matter the fitting method I use.

I am interested in identifying where the difference comes from and with what parameters i can (if at all) fit the ARIMA to get the same coefficients of the fit as from the ARMA.

Sample code to demonstrate:

library(tseries)
set.seed(2)
#getting a time series manually
x<-c(1,2,1)
e<-c(0,0.3,-0.2)
n<-45
AR<-c(0.5,-0.4,-0.1)
MA<-c(0.4,0.3,-0.2)
for(i in 4:n){
tt<-rnorm(1)
t<-x[length(x)]+tt+x[i-1]*AR[1]+x[i-2]*AR[2]+x[i-3]*AR[3]+e[i-1]*MA[1]+e[i-2]*MA[2]+e[i-3]*MA[3]
x<-c(x,t)
e<-c(e,tt)
}
par(mfrow=c(2,1))
plot(x)
plot(diff(x,1))

#fitting different versions. What I would like to get is fit1 with ARIMA()
fit1<-arma(diff(x,1,lag=1),c(3,3),include.intercept=F)
fit2<-arima(x,c(3,1,3),include.mean=F)
fit3<-arima(diff(x,1),c(3,0,3),include.mean=F)
fit4<-arima(x,c(3,1,3),method="CSS",include.mean=F)
fit5<-arima(diff(x,1),c(3,0,3),method="CSS",include.mean=F)

cbind(fit1$coe,fit2$coe,fit3$coe,fit4$coe,fit5$coe)

Edit: Using the conditional sum of squares comes pretty close, but is not quite there. Thanks for the hint for the fit1!

Edit2: I do not think this is a duplicate. Points 2 and 3 address different problems than mine, and even if I override the initialisation mentioned in point 1 by

fit4<-arima(x,c(3,1,3),method="CSS",include.mean=F,init=fit1$coe)

I still get different coefficients

Answer 1

There are three minor issues in tseries::arma compared to stats::arima that lead to a slightly different result in the ARMA model for the differenced series using tseries::arma and ARIMA in stats::arima.

• Starting values of the coefficients: stats::arima sets the initial AR and MA coefficients to zero, while tseries::arma uses the procedure described in Hannan and Rissanen (1982) is employed to get initial values of the coefficients.

• Scale of the objective function: the objective function in tseries::arma returns the value of the conditional sums of squares, RSS; stats::arima returns 0.5*log(RSS/(n-ncond)).

• Optimization algorithm: By default, Nelder-Mead is used in tseries::arma, while stats::arima employs the BFGS algorithm.

The last one can be changed through the argument optim.method in stats::arima but the others would require modifying the code. Below, I show an abridged version of the source code (minimal code for this particular model) for stats::arima where the three issues mentioned above are modified so that they are the same as in tseries::arma. After addressing these issues, the same result as in tseries::arma is obtained.

---

Minimal version of stats::arima (with the changes mentioned above):

# objective function, conditional sum of squares
# adapted from "armaCSS" in stats::arima
armaCSS <- function(p, x, arma, ncond)
{
  # this does nothing, except returning the vector of coefficients as a list
  trarma <- .Call(stats:::C_ARIMA_transPars, p, arma, FALSE)
  res <- .Call(stats:::C_ARIMA_CSS, x, arma, trarma[[1L]], trarma[[2L]], as.integer(ncond), FALSE)
  # return the conditional sum of squares instead of 0.5*log(res), 
  # actually CSS is divided by n-ncond but does not relevant in this case
  #0.5 * log(res)
  res
}
# initial values of coefficients  
# adapted from function "arma.init" within tseries::arma
arma.init <- function(dx, max.order, lag.ar=NULL, lag.ma=NULL)
{
  n <- length(dx)
  k <- round(1.1*log(n))
  e <- as.vector(na.omit(drop(ar.ols(dx, order.max = k, aic = FALSE, demean = FALSE, intercept = FALSE)$resid)))
      ee <- embed(e, max.order+1)
      xx <- embed(dx[-(1:k)], max.order+1)
      return(lm(xx[,1]~xx[,lag.ar+1]+ee[,lag.ma+1]-1)$coef) 
}
# modified version of stats::arima
modified.arima <- function(x, order, seasonal, init)
{
  n <- length(x)
  arma <- as.integer(c(order[-2L], seasonal$order[-2L], seasonal$period, order[2L], seasonal$order[2L]))
      narma <- sum(arma[1L:4L])
      ncond <- order[2L] + seasonal$order[2L] * seasonal$period
      ncond1 <- order[1L] + seasonal$period * seasonal$order[1L]
      ncond <- as.integer(ncond + ncond1)
      optim(init, armaCSS, method = "Nelder-Mead", hessian = TRUE, x=x, arma=arma, ncond=ncond)$par
}

Now, compare both procedures and check that yield the same result (requires the series x generated by the OP in the body of the question).

Using the initial values chosen in tseries::arima:

dx <- diff(x)
fit1 <- arma(dx, order=c(3,3), include.intercept=FALSE)
coef(fit1)
#         ar1         ar2         ar3         ma1         ma2         ma3 
#  0.33139827  0.80013071 -0.45177254  0.67331027 -0.14600320 -0.08931003 
init <- arma.init(diff(x), 3, 1:3, 1:3)
fit2.coef <- modified.arima(x, order=c(3,1,3), seasonal=list(order=c(0,0,0), period=1), init=init)
fit2.coef
# xx[, lag.ar + 1]1 xx[, lag.ar + 1]2 xx[, lag.ar + 1]3 ee[, lag.ma + 1]1 
#        0.33139827        0.80013071       -0.45177254        0.67331027 
# ee[, lag.ma + 1]2 ee[, lag.ma + 1]3 
#       -0.14600320       -0.08931003 
all.equal(coef(fit1), fit2.coef, check.attributes=FALSE)
# [1] TRUE

Using the initial values chosen in stats::arima (zeros):

fit3 <- arma(dx, order=c(3,3), include.intercept=FALSE, coef=rep(0,6))
coef(fit3)
#         ar1         ar2         ar3         ma1         ma2         ma3 
#  0.33176424  0.79999112 -0.45215742  0.67304072 -0.14592152 -0.08900624 
init <- rep(0, 6)
fit4.coef <- modified.arima(x, order=c(3,1,3), seasonal=list(order=c(0,0,0), period=1), init=init)
fit4.coef
# [1]  0.33176424  0.79999112 -0.45215742  0.67304072 -0.14592152 -0.08900624
all.equal(coef(fit3), fit4.coef, check.attributes=FALSE)
# [1] TRUE

Answer 2

As far as I can tell, the difference is entirely due to the MA terms. That is, when I fit your data with only AR terms, the ARMA of the differenced series and ARIMA agree.