Skip to main content
2 of 12
added 26 characters in body
Antoni Parellada
  • 26.9k
  • 18
  • 122
  • 230

I don't intend to be accurate, but to give a mental image... The quotes are from the link in the OP:

Identification of an AR model is often best done with the PACF.

For an AR model, the theoretical PACF “shuts off” past the order of the model. The phrase “shuts off” means that in theory the partial autocorrelations are equal to 0 beyond that point. Put another way, the number of non-zero partial autocorrelations gives the order of the AR model. By the “order of the model” we mean the most extreme lag of x that is used as a predictor.

... a $k^{\text{th}}$ order autoregression, written as AR(k), is a multiple linear regression in which the value of the series at any time t is a (linear) function of the values at times $t-1,t-2,\ldots,t-k:$

$$\begin{equation*} y_{t}=\beta_{0}+\beta_{1}y_{t-1}+\beta_{2}y_{t-2}+\cdots+\beta_{2}y_{t-2}\epsilon_{t}. \end{equation*}$$

Oh, but this looks like a regression model, but this was on the linked paged... So what is a possible intuition of what we are doing here...

In Chinese whispers or the telephone game as illustrated here

enter image description here

the message gets distorted as it is whispered from person to person, and all traces of resemblance (any truthful words, if you will) are lost after the red participant (with the exception of the article 'a'). PACF would tell us that the coefficients for the blue and the yellow participants are non-contributory once the effect of the brown and red participants are accounted for.

It is not difficult to come very close to the actual output of the R function by actually obtaining consecutive OLS regressions through the origin of farther lagged sequences, and collecting the coefficients into a vector. Schematically,enter image description here

a very similar process to the telephone game - it'll come a point, when there won't be any variability in the signal of the actual initial time series found in progressively more distant snippets of itself.


Identification of an MA model is often best done with the ACF rather than the PACF.

For an MA model, the theoretical PACF does not shut off, but instead tapers toward 0 in some manner. A clearer pattern for an MA model is in the ACF. The ACF will have non-zero autocorrelations only at lags involved in the model.

A moving average term in a time series model is a past error (multiplied by a coefficient).

The $q^{\text{th}}$-order moving average model, denoted by MA(q) is

$$x_t = \mu + w_t +\theta_1w_{t-1}+\theta_2w_{t-2}+\dots + \theta_qw_{t-q}$$

with $w_t \overset{iid}{\sim} N(0, \sigma^2_w).$

Here, it is not the message resemblance across time points that is search backwards in time step-by-step, but rather the contribution of the noise, which I picture as the often massive deviations that a random walk can lead along the time line:

enter image description here

Here there are multiple, progressively offset sequences that are correlated, discarding any contribution of the intermediate steps. This would be the graph of the operations involved:enter image description here

In this regard, "CV is cool!" is not completely different than "Naomi has a pool". From the noise point of view, the rhymes are still there all the way to the beginning of the game.

Antoni Parellada
  • 26.9k
  • 18
  • 122
  • 230