1
$\begingroup$

I'm trying to understand the steps in Rob Hyndman's Multi-step forecasts without re-estimation example below. I'm wondering what the purpose is of

refit <- Arima(x, model=fit)

The model has already been determined and trained by auto.arima in the "fit" step. So in the "refit" step are we re-training the model on a new data set? If so, what is the point of retraining the same model on a new data set?

url: http://robjhyndman.com/hyndsight/rolling-forecasts/

Code:

library(fpp)

h <- 5
train <- window(hsales,end=1989.99)
test <- window(hsales,start=1990)
n <- length(test) - h + 1
fit <- auto.arima(train)
fc <- ts(numeric(n), start=1990+(h-1)/12, freq=12)
for(i in 1:n)
{  
  x <- window(hsales, end=1989.99 + (i-1)/12)
  refit <- Arima(x, model=fit)
  fc[i] <- forecast(refit, h=h)$mean[h]
}

Updated Code to re-estimate coefficients:

h <- 5
train <- window(hsales,end=1989.99)
test <- window(hsales,start=1990)
n <- length(test) - h + 1
fit <- auto.arima(train)
order <- arimaorder(fit)
fc <- ts(numeric(n), start=1990+(h-1)/12, freq=12)
for(i in 1:n)
{  
  x <- window(hsales, end=1989.99 + (i-1)/12)
  refit <- Arima(x, order=order[1:3],seasonal=order[4:6])
  fc[i] <- forecast(refit, h=h)$mean[h]
}
$\endgroup$
2
$\begingroup$

No, we don't re-train the model. Here is what the help page ?Arima say for the model parameter:

If model is passed, this same model is fitted to ‘x’ without re-estimating any parameters.

Here is an example:

# library(forecast)
# model.1 <- auto.arima(AirPassengers[1:24])
# model.1
Series: AirPassengers[1:24] 
ARIMA(1,0,1) with non-zero mean 

Coefficients:
         ar1     ma1  intercept
      0.4137  0.6353   133.3991
s.e.  0.2091  0.1479     6.2032

sigma^2 estimated as 129.4:  log likelihood=-93
AIC=193.99   AICc=196.1   BIC=198.7

# model.2 <- Arima(AirPassengers[1:48],model=model.1)
# model.2
Series: AirPassengers[1:48] 
ARIMA(1,0,1) with non-zero mean 

Coefficients:
         ar1     ma1  intercept
      0.4137  0.6353   133.3991
s.e.  0.0000  0.0000     0.0000

sigma^2 estimated as 385.6:  log likelihood=-211.61
AIC=425.22   AICc=425.31   BIC=427.09

We note:

  • The estimated coefficients are the same. (No surprise, since they are not re-estimated.)
  • The standard errors are all zero. (I'd assume they are manually set this way, since they don't make any sense and would not be connected to the new data.)
  • The estimated residual variance $\sigma^2$ has changed. This makes sense, since prediction intervals for non-re-restimated parameters will be larger than for re-estimated parameters, since the parameters don't fit as well as re-estimated ones would have.
  • The log-likelihood and information criteria change, since they are all related to $\sigma^2$.

Now, if we forecast, we of course get different values, since in each case the last observations we autoregress on are different:

# forecast(model.1,h=6)$mean
Time Series:
Start = 25 
End = 30 
Frequency = 1 
[1] 150.1410 140.3254 136.2646 134.5846 133.8896 133.6020

# forecast(model.2,h=6)$mean
Time Series:
Start = 49 
End = 54 
Frequency = 1 
[1] 187.6868 155.8583 142.6906 137.2431 134.9894 134.0570

As to why we would not re-estimate the model after getting new data... I also don't see a really good reason. Perhaps in specific situations you might have performance issues. You may assume that a few more data points won't change the parameters a lot, especially if you already have a long time series with thousands of observations - in which case re-estimating would take some time, too.

$\endgroup$
  • $\begingroup$ Thank you for your answer. So if I understand correctly it sounds like "refit" isn't re-training the model. It's evaluating some of the measures of how well the model fits a new time-series (sigma^2, loglikelyhood...) Is there a way to train the same model (in this example an ARIMA(1,0,1) model) but on a new time-series and refit the foeficients? Is re-training the same ARIMA(1,0,1) on new samples from the same time-series likely to improve accuracy? $\endgroup$ – user3476463 Apr 21 '16 at 16:36
  • $\begingroup$ You can re-estimate the coefficients by specifying the order parameter instead of the model one, e.g., ` Arima(AirPassengers[1:48],order=c(1,0,1))`. And yes, I would expect re-estimating with more data to yield a more accurate forecast. (With more data, we may even want to switch models.) $\endgroup$ – S. Kolassa - Reinstate Monica Apr 21 '16 at 19:26
  • $\begingroup$ I've added updated code to my original post that is an attempt to retrain the model which is determined in the "fit" step. Will the updated code retrain the model and re-estimate the coefficients like you described in your previous post? $\endgroup$ – user3476463 Apr 22 '16 at 1:33
  • $\begingroup$ Mostly yes. However, if the model is not seasonal, then arimaorder will only return a vector of length 3, so extracting entries 4:6 will break. You may need a case distinction on length(order). (Plus, order() is an R command. It's not a good idea to overwrite existing commands. You may at some point want to use order() and run into very hard-to-understand bugs.) $\endgroup$ – S. Kolassa - Reinstate Monica Apr 22 '16 at 14:50
0
$\begingroup$

This problem occur in the following scenario. Assume that you use data from until one week ago to train the model. Now you want to use this model to make prediction for tomorrow, but you don't want to retrain the whole model as this take a lot of time. Arima will help you with this kind of problem.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.