2
$\begingroup$

I am really trying, but struggling, to understand how Autoregressive and Moving Average work. I am pretty terrible with algebra and looking at it doesn't really improve my understanding of something. What I would really love is an extremely simple example of say 10 time dependent observations so I can 'see' how they work. So let's say you have the following data points of the price of gold:

Time Gold Price ($)
1       4
2       6
3       6
4       8
5       7
6       6
7       4
8       3
9       3
10      4

For example, at time period 10, what would the Moving Average of Lag 2, MA(2), be? Or MA(1)? And AR(1) or AR(2)?

I traditionally learned about Moving Average being something like:

(sum of n observations)/n

But when looking at ARMA models, MA is explained as a function of previous error terms, which I can't get my head around. Is it just a fancier way of calculating the same thing?

I found this post helpful: (How to understand SARIMAX intuitively?) but whist the algebra helps, I can't see something really clearly until I see a simplified example of it.

$\endgroup$
  • $\begingroup$ A side note: algebra may not be very transparent in ARMA models of higher order. Impulse-response analysis helps visualize the working of the model quite well. You may benefit from familiarizing yourself with it. $\endgroup$ – Richard Hardy Aug 13 '15 at 19:55
  • $\begingroup$ Given the gold price data, you would first estimate the model and then see how it works (impulse-response analysis; forecasts). Perhaps you should narrow down your question to just the second part (and leave estimation aside). That is, you would provide an AR(1) or MA(1) or whatever model (e.g. $x_t=0.5 x_{t-1}+\varepsilon_t$) and ask us, how does this particular model work. $\endgroup$ – Richard Hardy Aug 13 '15 at 19:58
3
$\begingroup$

For any AR(q) model the easy way to estimate the parameter(s) is to use OLS - and run the regression of:

$$\newcommand{\price}{{\rm price}} \price_t = \beta_0 + \beta_1 \cdot \price_{t-1} \dotso \beta_q \cdot \price_{t-q} $$

Lets do so (in R):

# time series setup:
price <- c(4, 6, 6, 8, 7, 6, 4, 3, 3, 4)
price <- ts(data=price, start=1, end=10)

# Estimate AR(1) parameter:
ar1 <- arima(price, order=c(  1   # AR order
                            , 0   # Integration order
                            , 0)) # MA order

# The AR(1) parameter is 0.6608, with mean of 4.7916
# such that the intercept is 4.7916*(1-0.6608) = 1.625311

# The fitted value (t = 2) is then:
1.625311 + 4 * 0.6608 # 4.26
# t = 3:
1.625311 + 6 * 0.6608 # 5.59
# ... up too: t = 10
1.625311 + 3 * 0.6608 # 3.61

# Actually R can calculate all of these
# require(forecast)
# fitted(ar1), which yields the same.

(Okay, so I cheated a bit and used the arima function in R, but it yields the same estimates as the OLS regression - try it).

Now lets have a look at the MA(1) model. Now the MA model is very different from the AR model. The MA is weighted average of past periods error, where as the AR model uses the previoues periods actual data values. The MA(1) is:

$$ \price_t = \mu + w_t + \theta_1 \cdot w_{t-1} $$

Where $\mu$ is the mean, and $w_t$ are the error terms - not the previous value of price (as in the AR model). Now, alas, we can't estimate the parameters by something as simple as OLS. I will not cover the method here, but the R function arima uses maximum likelihood. Lets try:

# Estimate MA(1) parameter:
ma1 <- arima(price, order=c(  0   # AR order
                            , 0   # Integration order
                            , 1)) # MA order
# MA(1) parameter is: 0.5423, and the mean is 4.9977
# Now we need the residuals, for the first period it is 0 
# For the second period it is:
6 - (0.5423 * 0 + 4.9977) = 1.0023

# So for the second period the forecast is:
4.9977 + 0 + .5323 * 1.0023

# Now you can calculate the rest... 
# Excel might be a better choice if you want to see what is going on.
$\endgroup$
  • $\begingroup$ Many thanks for this. A few questions. (1) I tried to do an OLS for the AR(1) by doing something like lm(price ~ zlag(price, 1)) but I got a coeffiecient of 0.687 and an intercept of 1.635. These are slightly different from the answers you gave above. Have I done it right? $\endgroup$ – Will T-E Aug 17 '15 at 11:17
  • $\begingroup$ (2) Regarding the MA(1) question. You say the residual is 1.0023 for the second period. That makes sense. My understanding of the residual is it's the difference between the forecasted value and the observed value. But you then say the forecasted value for period 2, is calculated using the residual for period 2. Is that right? Isn't the forecasted value for period 2 just (0.5423*0 + 4.9977)? $\endgroup$ – Will T-E Aug 17 '15 at 11:24
  • $\begingroup$ (3) What I also still can't get my head around is why is it called Moving Average? The Moving average I traditionally learned about was a mean value of the last n observed values. The MA(1) here doesn't seem to be much of an average, but just takes a slice of the last error/residual. $\endgroup$ – Will T-E Aug 17 '15 at 11:47
  • $\begingroup$ @WillT-E; Another way to think about the MA(q) model; en.wikipedia.org/wiki/Moving_average#Simple_moving_average $\endgroup$ – Repmat Aug 17 '15 at 18:57
  • $\begingroup$ Any thoughts on the comments above? Particularly the first two. $\endgroup$ – Will T-E Aug 20 '15 at 10:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.