# Predict from estimated ARIMA model with new data

Suppose I have a training dataset, I use auto.arima (from "forecast" package in R) to fit the training data. As a result I get the lag and integration orders $(p, d, q)$ and the corresponding coefficients $\psi_i$ and $\theta_i$.

ytrain = c(0.435477843, 0.435394762, 0.195528995, 1.451623315, 1.740084831 2.379904714, 1.092366508, 0.001031411, 0.592164090, 0.670323418)

fit <- auto.arima(ytrain)


Now I have new data

ytest = c(-0.1349199  0.9001208 -0.5171740 -0.9958452  0.4125953 -0.3320575  0.1633313  0.2890109 -0.4284824  0.7902680)


I want to fit this new data by using the model from training data (using the same $(p, d, q)$ and also the same corresponding coefficients). I.e. I want to use the model I have from ytrain to make prediction based on ytest. As a result I can know if there are any points in the new data looking like anomaly points (compared to the training data)

I have searched long time and haven't find a R function to implement it. I know I can compute this by hand, e.g. for ARMA(1,2):

$$\hat{Y}_n = \hat{\mu} + \hat{\psi}_1 Y_{n-1} - \hat{\theta}_{1} \epsilon_{n-1} - \hat{\theta}_2 \epsilon_{n-2}.$$

But if I do this, I am not sure how to start to get $\epsilon_1 = Y_1 - \hat{Y}_1$ and $\epsilon_2 = Y_2 - \hat{Y}_2$ to start since I don't have $\hat{Y}_1$ and $\hat{Y}_2$.

• Could anyone suggest an R function for doing this? Or if not,
• Could anyone help me with this question if there is no R function doing this?
• I was going through my old answers and noticed this one was not accepted. Do you perhaps need further clarification? – Richard Hardy Feb 19 '17 at 8:56

I suppose you are not looking for multiple-step-ahead forecasts from the model estimated on the original data set (this is very easy to do with forecast.Arima or predict; you should have been able to figure that out by yourself).

I suppose you want to incorporate the new values as they come (but without reestimating the model) and use them in place of predicted values to produce one-step-ahead forecasts.

Using your own example, I suppose you do not want

$$\hat{Y}_{t+1} = \hat{\mu} + \hat{\psi}_1 \hat Y_{t} - \hat{\theta}_{1} \hat\epsilon_{t} - \hat{\theta}_2 \hat\epsilon_{t-1}$$

but rather (check out tildes and hats carefully)

$$\tilde{Y}_{t+1} = \hat{\mu} + \hat{\psi}_1 Y_{t} - \hat{\theta}_{1} \tilde\epsilon_{t} - \hat{\theta}_2 \hat\epsilon_{t-1}$$

where $\hat\epsilon_{t}=0$ (based on model prediction not using the new value $Y_t$) while $\tilde\epsilon_{t}=Y_t-\hat Y_t$ (based on the actual value versus a prediction).

This thread tells you how to refit your model on completely new data: "How to use a fitted model parameters for forecasting other time series".

Based on that idea, you can

1. append the original data set with the new observations, one at a time;
2. refit the model (without reestimating it);

Also, if you are doing things by hand, the model fitted on a sample spanning $1,\dotsc,T$ gives you fitted values $\hat Y$ and $\hat\epsilon$ all the way up to $T$, so obtaining them should be no problem.

• You're maybe right, but he said "I want to fit this new data by using the model from training data (using the same (p,d,q)(p,d,q) and also the same corresponding coefficients). I.e. I want to use the model I have from ytrain to make prediction based on ytest". So .. – el Josso Jul 13 '16 at 7:43
• @elJosso, you might be right. But that is only the better for you as you have posted an answer based on that idea. Let us wait until the OP reacts, and I will happily upvote your answer if that is what he actually needed. – Richard Hardy Jul 13 '16 at 7:54
• Thanks for the help. Really appreciate it. I have looked at the thread you give me. But I am not sure what is this "refit <- Arima(newdata, model=fit) " actually doing. Does it fit new data with same model but find different coefficients? Maybe I haven't phrased my question clearly. What I am doing is to detect anomaly segment of new data using ARIMA method. Given training data, I got the model (p, d, q) and coefficients. Given another test data, I used (p,d,q) and coefficients from the previous model to refit the data within the range not prediction. – user123109 Jul 13 '16 at 14:29
• @Richard Hardy is correct. But the index t should be from 1 to n, instead of n+1. I suppose the meaning of tilde is fitting new data with information from training data. What I need is $\tilde{Y}_1$ ,.... $\tilde{Y}_n$ conditional on p,d,q $\psi_i$ $\theta_i$ from training data. – user123109 Jul 13 '16 at 14:33
• Suppose I want to do it by hand. For ARMA(p, q) model. Suppose $w = max(p,q)$, I need $\epsilon_1$,...,$\epsilon_w$ to find fitted value from $\hat{Y}_{w+1}$,... $\hat{Y}_n$. How to initialize these $\epsilon_1$,...,$\epsilon_w$? I have tried set them to zero, but the result is different from fitted() from R. – user123109 Jul 13 '16 at 14:39

I think you are looking for this method, aren't you ?

## An example of ARIMA forecasting ( with ARIMA from package Stats):

predict(fit, 10)

## An example of ARIMA forecasting ( with ARIMA from package forecast):

forecast(fit,h=10) forecast(fit,10)\$fitted

So you are forecasting 10 steps on your model

When you are using a package think about read the doc : https://cran.r-project.org/web/packages/forecast/forecast.pdf.
There is a LOT of informaton inside