1
$\begingroup$

I have been working with the forecast package in R a lot, recently. And my question might seem trivial (or not, maybe I'm missing something), but for the life of me I can't seem to find a way to fit an Arima model with exogenous variables (xreg argument) that has been computed by the auto.arima function to previously unseen test data.

So, I'm basically trying to do the following:

library(forecast)
fit <- auto.arima(trainingdata, xreg = trainingvariables)

...and then I would like to "apply" the model to new test data, for which I also have new exogenous variables available. I can see the following methods:

fitted(fit)

That returns one-step in-sample forecasts, so, in effect, that's exactly what I want. Except that it's in-sample. However, I would like to calculate one-step out-of-sample forecasts (with new exogenous variables that I have available). Another method:

forecast(fit, xreg = newvariables, h = ...)

That works for exactly one step, but then seems to merely forecast the trainingdata stored in the model fit. But I don't think I can use new testdata here? (So, I can't use this method for testing one-step prediction accuracy.) One more idea:

fit2 <- Arima(testdata, model = fit)

According to the manual, if the model parameter is used, "this same model is fitted to [testdata] without re-estimating any parameters". Great, but I don't think I can supply any new exogenous variables, can I?

I really think, I must be missing something simple. Any help would be much appreciated.

|cite|improve this question
$\endgroup$
  • $\begingroup$ Be careful about terminology and model definitions. If you are using auto.arima, you are fitting regressions with ARMA errors rather than ARIMAX model; see Rob J. Hyndman's blog post "The ARIMAX model muddle". $\endgroup$ – Richard Hardy Jan 27 '16 at 15:03
  • $\begingroup$ Yes, that's true, of course, especially when using exogenous variables (xreg). I'm not sure, though, what name to give to what the auto.arima function returns... In my workflow it makes sense to also call this structure a "model", but I do realise that that might not be correct in a strict statistical sense. $\endgroup$ – Marcus C. Jan 27 '16 at 15:16
  • $\begingroup$ I didn't mean you should call it something else than "a model"; I only intended to point out the differences between regression with ARMA errors, ARIMAX model, etc. $\endgroup$ – Richard Hardy Jan 27 '16 at 15:29
  • $\begingroup$ This may be helpful: R time-series forecasting with auto.arima and xreg=explanatory variables. If not, consider asking at StackOverflow, but provide a minimal working example. I assume your newvariables may not be a ts or a data.frame, but we have no way of ascertaining this. $\endgroup$ – S. Kolassa - Reinstate Monica Jan 27 '16 at 15:42
  • $\begingroup$ Thanks for your help. Unfortunately, that link isn't really helpful, as it deals with forecasting a fitted timeseries into the future (30 steps in the given example). What I would like to achieve, is the same thing as the fitted function (repeated one-step forecasting), but not in-sample, but out-of-sample. I would simply like to plot both the (one-step) fitted and the correct time series on top of each other. $\endgroup$ – Marcus C. Jan 27 '16 at 15:56
2
$\begingroup$

I am not sure that I understand correctly what you are asking, but if you just want to plot one-step forecasts of a model with a train set and new test set here is a reproducible example:

library(forecast)

# Number of observations
n <- 100

# Generate train set
wn <- rnorm(n)
x <- rnorm(n)
y <- numeric(n)
y[1] <- rnorm(1)
for (i in 2:n) {
  y[i] <- 0.2 + 0.5 * y[i-1] + 0.7 * x[i] + wn[i]
}

# Generate new test data
wn2 <- rnorm(n)
x2 <- rnorm(n)
y2 <- numeric(n)
y2[1] <- rnorm(1)
for (i in 2:n) {
  y2[i] <- 0.2 + 0.5 * y2[i-1] + 0.7 * x2[i] + wn2[i]
}

# Fit model to train data
fit <- Arima(y, order = c(1,0,0), xreg = x)

# Fit same model to test data
fit2 <- Arima(y2, model = fit, xreg = x2)

# Plot fitted values
plot(fitted(fit))
lines(fitted(fit2), col = "blue")

If this is not what you wanted just add a comment that I will remove the answer and wait for more detailed explanations in the question.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Yes, thank you very much! In your example I replaced the line plot(fitted(fit)) by plot(y2, type="l"), but otherwise that was exactly, what I was hoping for. It seems, I completely missed the fact that I can add a new xreg parameter, when fitting the trained model to the test data. Thanks again, much appreciated. $\endgroup$ – Marcus C. Jan 27 '16 at 16:58

Not the answer you're looking for? Browse other questions tagged or ask your own question.