In R (2.15.2) I fitted once an ARIMA(3,1,3) on a time series and once an ARMA(3,3) on the once differenced timeseries. The fitted parameters differ, which I attributed to the fitting method in ARIMA.

Also, fitting an ARIMA(3,0,3) on the same data as ARMA(3,3) will not result in identical parameters, no matter the fitting method I use.

I am interested in identifying where the difference comes from and with what parameters i can (if at all) fit the ARIMA to get the same coefficients of the fit as from the ARMA.

Sample code to demonstrate:

library(tseries)
set.seed(2)
#getting a time series manually
x<-c(1,2,1)
e<-c(0,0.3,-0.2)
n<-45
AR<-c(0.5,-0.4,-0.1)
MA<-c(0.4,0.3,-0.2)
for(i in 4:n){
tt<-rnorm(1)
t<-x[length(x)]+tt+x[i-1]*AR[1]+x[i-2]*AR[2]+x[i-3]*AR[3]+e[i-1]*MA[1]+e[i-2]*MA[2]+e[i-3]*MA[3]
x<-c(x,t)
e<-c(e,tt)
}
par(mfrow=c(2,1))
plot(x)
plot(diff(x,1))

#fitting different versions. What I would like to get is fit1 with ARIMA()
fit1<-arma(diff(x,1,lag=1),c(3,3),include.intercept=F)
fit2<-arima(x,c(3,1,3),include.mean=F)
fit3<-arima(diff(x,1),c(3,0,3),include.mean=F)
fit4<-arima(x,c(3,1,3),method="CSS",include.mean=F)
fit5<-arima(diff(x,1),c(3,0,3),method="CSS",include.mean=F)

cbind(fit1$coe,fit2$coe,fit3$coe,fit4$coe,fit5$coe)

Edit: Using the conditional sum of squares comes pretty close, but is not quite there. Thanks for the hint for the fit1!

Edit2: I do not think this is a duplicate. Points 2 and 3 address different problems than mine, and even if I override the initialisation mentioned in point 1 by

fit4<-arima(x,c(3,1,3),method="CSS",include.mean=F,init=fit1$coe)

I still get different coefficients

  • fit1 has only 1 MA & 1 AR parameter: did you mean fit1<-arma(diff(x,1,lag=1),c(3,3),include.intercept=F)? – Scortchi Dec 6 '13 at 12:45
  • 1
    I'd assume there's some slight difference in the fitting algorithms even when you specify minimizing the conditional sum of squared errors. The help pages for arima mention an n.cond argument giving the number of observations at the start of the series to ignore when calculating it - perhaps that's it. (What's wrong with using maximum likelihood anyway?) – Scortchi Dec 6 '13 at 15:24
  • AFAIK n.cond does not use the first few observations to fit. It did not help me there. Nothing wrong with ML at all. I just would like to understand the differences. – user1965813 Dec 10 '13 at 6:30
  • 3
    Duplicate? stats.stackexchange.com/a/32799/159 – Rob Hyndman Jan 10 '14 at 23:47
up vote 10 down vote accepted

There are three minor issues in tseries::arma compared to stats::arima that lead to a slightly different result in the ARMA model for the differenced series using tseries::arma and ARIMA in stats::arima.

  • Starting values of the coefficients: stats::arima sets the initial AR and MA coefficients to zero, while tseries::arma uses the procedure described in Hannan and Rissanen (1982) is employed to get initial values of the coefficients.

  • Scale of the objective function: the objective function in tseries::arma returns the value of the conditional sums of squares, RSS; stats::arima returns 0.5*log(RSS/(n-ncond)).

  • Optimization algorithm: By default, Nelder-Mead is used in tseries::arma, while stats::arima employs the BFGS algorithm.

The last one can be changed through the argument optim.method in stats::arima but the others would require modifying the code. Below, I show an abridged version of the source code (minimal code for this particular model) for stats::arima where the three issues mentioned above are modified so that they are the same as in tseries::arma. After addressing these issues, the same result as in tseries::arma is obtained.


Minimal version of stats::arima (with the changes mentioned above):

# objective function, conditional sum of squares
# adapted from "armaCSS" in stats::arima
armaCSS <- function(p, x, arma, ncond)
{
  # this does nothing, except returning the vector of coefficients as a list
  trarma <- .Call(stats:::C_ARIMA_transPars, p, arma, FALSE)
  res <- .Call(stats:::C_ARIMA_CSS, x, arma, trarma[[1L]], trarma[[2L]], as.integer(ncond), FALSE)
  # return the conditional sum of squares instead of 0.5*log(res), 
  # actually CSS is divided by n-ncond but does not relevant in this case
  #0.5 * log(res)
  res
}
# initial values of coefficients  
# adapted from function "arma.init" within tseries::arma
arma.init <- function(dx, max.order, lag.ar=NULL, lag.ma=NULL)
{
  n <- length(dx)
  k <- round(1.1*log(n))
  e <- as.vector(na.omit(drop(ar.ols(dx, order.max = k, aic = FALSE, demean = FALSE, intercept = FALSE)$resid)))
      ee <- embed(e, max.order+1)
      xx <- embed(dx[-(1:k)], max.order+1)
      return(lm(xx[,1]~xx[,lag.ar+1]+ee[,lag.ma+1]-1)$coef) 
}
# modified version of stats::arima
modified.arima <- function(x, order, seasonal, init)
{
  n <- length(x)
  arma <- as.integer(c(order[-2L], seasonal$order[-2L], seasonal$period, order[2L], seasonal$order[2L]))
      narma <- sum(arma[1L:4L])
      ncond <- order[2L] + seasonal$order[2L] * seasonal$period
      ncond1 <- order[1L] + seasonal$period * seasonal$order[1L]
      ncond <- as.integer(ncond + ncond1)
      optim(init, armaCSS, method = "Nelder-Mead", hessian = TRUE, x=x, arma=arma, ncond=ncond)$par
}

Now, compare both procedures and check that yield the same result (requires the series x generated by the OP in the body of the question).

Using the initial values chosen in tseries::arima:

dx <- diff(x)
fit1 <- arma(dx, order=c(3,3), include.intercept=FALSE)
coef(fit1)
#         ar1         ar2         ar3         ma1         ma2         ma3 
#  0.33139827  0.80013071 -0.45177254  0.67331027 -0.14600320 -0.08931003 
init <- arma.init(diff(x), 3, 1:3, 1:3)
fit2.coef <- modified.arima(x, order=c(3,1,3), seasonal=list(order=c(0,0,0), period=1), init=init)
fit2.coef
# xx[, lag.ar + 1]1 xx[, lag.ar + 1]2 xx[, lag.ar + 1]3 ee[, lag.ma + 1]1 
#        0.33139827        0.80013071       -0.45177254        0.67331027 
# ee[, lag.ma + 1]2 ee[, lag.ma + 1]3 
#       -0.14600320       -0.08931003 
all.equal(coef(fit1), fit2.coef, check.attributes=FALSE)
# [1] TRUE

Using the initial values chosen in stats::arima (zeros):

fit3 <- arma(dx, order=c(3,3), include.intercept=FALSE, coef=rep(0,6))
coef(fit3)
#         ar1         ar2         ar3         ma1         ma2         ma3 
#  0.33176424  0.79999112 -0.45215742  0.67304072 -0.14592152 -0.08900624 
init <- rep(0, 6)
fit4.coef <- modified.arima(x, order=c(3,1,3), seasonal=list(order=c(0,0,0), period=1), init=init)
fit4.coef
# [1]  0.33176424  0.79999112 -0.45215742  0.67304072 -0.14592152 -0.08900624
all.equal(coef(fit3), fit4.coef, check.attributes=FALSE)
# [1] TRUE
  • Great Work. Thank you very much! For me I added a tolerance argument to also be able to compare your two solutions to the normal arima function and it all worked like a charm. Thanks a lot! – user1965813 May 7 '15 at 5:49

As far as I can tell, the difference is entirely due to the MA terms. That is, when I fit your data with only AR terms, the ARMA of the differenced series and ARIMA agree.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.