Simple Example of Autoregressive and Moving Average

Question

I am really trying, but struggling, to understand how Autoregressive and Moving Average work. I am pretty terrible with algebra and looking at it doesn't really improve my understanding of something. What I would really love is an extremely simple example of say 10 time dependent observations so I can 'see' how they work. So let's say you have the following data points of the price of gold:

Time Gold Price ($)
1       4
2       6
3       6
4       8
5       7
6       6
7       4
8       3
9       3
10      4

For example, at time period 10, what would the Moving Average of Lag 2, MA(2), be? Or MA(1)? And AR(1) or AR(2)?

I traditionally learned about Moving Average being something like:

(sum of n observations)/n

But when looking at ARMA models, MA is explained as a function of previous error terms, which I can't get my head around. Is it just a fancier way of calculating the same thing?

I found this post helpful: (How to understand SARIMAX intuitively?) but whist the algebra helps, I can't see something really clearly until I see a simplified example of it.

Answer 1

For any AR(q) model the easy way to estimate the parameter(s) is to use OLS - and run the regression of:

$$\newcommand{\price}{{\rm price}} \price_t = \beta_0 + \beta_1 \cdot \price_{t-1} \dotso \beta_q \cdot \price_{t-q} $$

Lets do so (in R):

# time series setup:
price <- c(4, 6, 6, 8, 7, 6, 4, 3, 3, 4)
price <- ts(data=price, start=1, end=10)

# Estimate AR(1) parameter:
ar1 <- arima(price, order=c(  1   # AR order
                            , 0   # Integration order
                            , 0)) # MA order

# The AR(1) parameter is 0.6608, with mean of 4.7916
# such that the intercept is 4.7916*(1-0.6608) = 1.625311

# The fitted value (t = 2) is then:
1.625311 + 4 * 0.6608 # 4.26
# t = 3:
1.625311 + 6 * 0.6608 # 5.59
# ... up too: t = 10
1.625311 + 3 * 0.6608 # 3.61

# Actually R can calculate all of these
# require(forecast)
# fitted(ar1), which yields the same.

(Okay, so I cheated a bit and used the arima function in R, but it yields the same estimates as the OLS regression - try it).

Now lets have a look at the MA(1) model. Now the MA model is very different from the AR model. The MA is weighted average of past periods error, where as the AR model uses the previoues periods actual data values. The MA(1) is:

$$ \price_t = \mu + w_t + \theta_1 \cdot w_{t-1} $$

Where $\mu$ is the mean, and $w_t$ are the error terms - not the previous value of price (as in the AR model). Now, alas, we can't estimate the parameters by something as simple as OLS. I will not cover the method here, but the R function arima uses maximum likelihood. Lets try:

# Estimate MA(1) parameter:
ma1 <- arima(price, order=c(  0   # AR order
                            , 0   # Integration order
                            , 1)) # MA order
# MA(1) parameter is: 0.5423, and the mean is 4.9977
# Now we need the residuals, for the first period it is 0 
# For the second period it is:
6 - (0.5423 * 0 + 4.9977) = 1.0023

# So for the second period the forecast is:
4.9977 + 0 + .5323 * 1.0023

# Now you can calculate the rest... 
# Excel might be a better choice if you want to see what is going on.