# How is the first residual calculated in a fitted AR(1) model?

I am trying to figure out how the first residual is calculated in an AR(1) model. It's easy to generate all of the other residuals, but I have no idea how r calculates the first one.

Here is an example that I am working with:

> set.seed(1)  #use 390
> x <- arima.sim(n = 20, model=list(order=c(1,0,0), ar=c(0.7)))
> fit <- arima(x, c(1,0,0), include.mean = F)
> residuals <- 0
> residuals[2:20] <- x[2:20] - fit$coef * x[1:19] > data.frame(residuals, fit$residuals)
residuals fit.residuals
1   0.00000000    0.99077920
2   0.56625275    0.56625275
3   0.88811131    0.88811131
4   0.74271680    0.74271680
5   0.03181057    0.03181057
6  -2.02072514   -2.02072514
7   0.63642551    0.63642551
8  -0.05652348   -0.05652348
9  -0.15498384   -0.15498384
10 -1.46716431   -1.46716431
11 -0.44712965   -0.44712965
12  0.44892420    0.44892420
13  1.37226611    1.37226611
14 -0.11961349   -0.11961349
15  0.37788599    0.37788599
16 -0.06816952   -0.06816952
17 -1.38607175   -1.38607175
18 -0.39461047   -0.39461047
19 -0.37197692   -0.37197692
20 -0.03605144   -0.03605144


Ultimately, I would like to get a clearer understanding of how forecasts are generated for ARIMA models. But, to forecast the MA portion, I need to know the residuals for all of the observed values in the series. Not understanding how to calculate the first residual thus poses an issue.

Thanks.

• Here is a related question. It has been answered. The essence is, maximum likelihood estimation gives you everything, including the first value (while least-squares based estimation would not give you the first value). – Richard Hardy Apr 24 '16 at 18:25

## 2 Answers

In stats::arima, the first residual of an AR(1) model is obtained as a byproduct of the Kalman filter. Example for an AR(1) model:

# generate 120 observations from an AR(1) model
set.seed(123)
y <- arima.sim(n = 120, model = list(ar=0.6))
# fit the model
fit <- arima(y, order = c(1,0,0), include.mean = FALSE)
# get the state space representaton of the fitted model and
# run the Kalman filter
ss <- makeARIMA(phi = coef(fit), theta = numeric(0), Delta = numeric(0))
kf <- KalmanRun(y = y, mod = ss)
# residuals
head(as.vector(residuals(fit)))
#   0.5017314 -0.5510861  1.7855220  0.5106597 -1.9551303  0.6932320
head(kf$resid) #  0.5017314 -0.5510861 1.7855220 0.5106597 -1.9551303 0.6932320 all.equal(as.vector(residuals(fit)), kf$resid)
#  TRUE


For some introduction on how the Kalman filter operates on ARMA models you may see, for example, this post.

Beane, please see my comment here for an explanation of how the first residual is calculated. The unscaled value, not returned by R, is simply equal to $y_{1}$. However, R scales the residual by the standardized one-step ahead forecast error, obtained through the Kalman Filter, and returns that instead of the raw residuals.

R's implementation of arima follows Durbin & Koopman very closely, so I believe in order to understand how the forecasts are actually generated this should be required reading. Luckily, the implementation relies only on a few critical sections, mostly in chapters 2 and 4, so you wouldn't need to read the whole book.

Also note that what we are calling residuals are referred to by Durbin & Koopman as one-step ahead prediction errors. They mean the same thing.