How is the first residual calculated in a fitted AR(1) model?

Question

I am trying to figure out how the first residual is calculated in an AR(1) model. It's easy to generate all of the other residuals, but I have no idea how r calculates the first one.

Here is an example that I am working with:

> set.seed(1)  #use 390
> x <- arima.sim(n = 20, model=list(order=c(1,0,0), ar=c(0.7)))
> fit <- arima(x, c(1,0,0), include.mean = F)
> residuals <- 0
> residuals[2:20] <- x[2:20] - fit$coef[1] * x[1:19]
    > data.frame(residuals, fit$residuals)
     residuals fit.residuals
1   0.00000000    0.99077920
2   0.56625275    0.56625275
3   0.88811131    0.88811131
4   0.74271680    0.74271680
5   0.03181057    0.03181057
6  -2.02072514   -2.02072514
7   0.63642551    0.63642551
8  -0.05652348   -0.05652348
9  -0.15498384   -0.15498384
10 -1.46716431   -1.46716431
11 -0.44712965   -0.44712965
12  0.44892420    0.44892420
13  1.37226611    1.37226611
14 -0.11961349   -0.11961349
15  0.37788599    0.37788599
16 -0.06816952   -0.06816952
17 -1.38607175   -1.38607175
18 -0.39461047   -0.39461047
19 -0.37197692   -0.37197692
20 -0.03605144   -0.03605144

Ultimately, I would like to get a clearer understanding of how forecasts are generated for ARIMA models. But, to forecast the MA portion, I need to know the residuals for all of the observed values in the series. Not understanding how to calculate the first residual thus poses an issue.

Thanks.

Answer 1

In stats::arima, the first residual of an AR(1) model is obtained as a byproduct of the Kalman filter. Example for an AR(1) model:

# generate 120 observations from an AR(1) model
set.seed(123)
y <- arima.sim(n = 120, model = list(ar=0.6))
# fit the model
fit <- arima(y, order = c(1,0,0), include.mean = FALSE)
# get the state space representaton of the fitted model and 
# run the Kalman filter
ss <- makeARIMA(phi = coef(fit), theta = numeric(0), Delta = numeric(0))
kf <- KalmanRun(y = y, mod = ss)
# residuals
head(as.vector(residuals(fit)))
# [1]  0.5017314 -0.5510861  1.7855220  0.5106597 -1.9551303  0.6932320
head(kf$resid)
# [1]  0.5017314 -0.5510861  1.7855220  0.5106597 -1.9551303  0.6932320
all.equal(as.vector(residuals(fit)), kf$resid)
# [1] TRUE

For some introduction on how the Kalman filter operates on ARMA models you may see, for example, this post.

Answer 2

Beane, please see my comment here for an explanation of how the first residual is calculated. The unscaled value, not returned by R, is simply equal to $y_{1}$. However, R scales the residual by the standardized one-step ahead forecast error, obtained through the Kalman Filter, and returns that instead of the raw residuals.

R's implementation of arima follows Durbin & Koopman very closely, so I believe in order to understand how the forecasts are actually generated this should be required reading. Luckily, the implementation relies only on a few critical sections, mostly in chapters 2 and 4, so you wouldn't need to read the whole book.

Also note that what we are calling residuals are referred to by Durbin & Koopman as one-step ahead prediction errors. They mean the same thing.