Autocorrelation and Partial Correlation plots in ARMA models Consider the following input and its Autocorrelation and Partial Autocorrelation plots (source). 


*

*What are the shaded blue areas in these plots?  I often see them when studying ARMA models. What do they represent?

*I have read the technical definition of the partial autocorrelation function, but I am still having a hard time relating it to the standard autocorrelation function, which I understand. What does PACF exactly measure, and why is it relevant for ARMA models?


 A: *

*The blue shaded part joins the boundaries of an approximate 95% interval for the individual correlations assuming the series is independent. So if your data were white noise, about 5% of those autocorrelations would be expected to lie outside those bounds.

*The PACF is basically the lagged correlations adjusted for the effect of lower order correlation. For example, if you have an AR(1) with autocorrelation at lag 1 $\rho_1$, then the correlation at lag 2 will be $\rho_1^2$. If you want to assess what's going on apart from the correlation you already expect at lag 2 because of the correlation at lag 1, you want the PACF at lag 2.
If you have a roughly sinusoidal series, you'll typically see a damped sinusoid in the ACF. But notice that your PACF cuts off ... so that ongoing sinusoid is largely due to the pattern of the earlier lags; a low order AR might describe the data reasonably well (but actually, looking at the data, I think it's not a simple AR as such; there's what looks like some periodic effect, and you should try to model that. Also the data seem to be positive, right skew and nonstationary, so you should beware trying to overinterpret your displays in terms of ARMA models.
A: The blue shaded areas are used to test the statistical significance of the autocorrelation and partial autocorrelation coefficients. In the ACF, these bands are sometimes based on Bartlett's standard errors, which go back to a paper published in 1946. They are calculated using the formula below, where $r_{k}$ denotes the kth estimated autocorrelation coefficient and $n$ is the number of observations of the time-series:
$$ (1 + 2 \sum_{j=1}^{k-1}r_{j}^{2})^{1/2} n^{-1/2}. $$
The estimated standard error plotted in the PACF, which is sometimes plotted in the ACF instead of Bartlett's errors, is calculated using the following formula:
$$ n^{-1/2}. $$
The ACF shows the correlation between ordered pairs separated by various time spans. For example, the correlation between $x_{t}$ and $x_{t-1}$, or, say, $x_{t}$ and $x_{t-2}$, and so on. 
The PACF measures the correlation between ordered pairs separated by various time spans taking into account the effects of intervening pairs. So, the PACF differs from the ACF in terms of accounting for intervening effects. For example, the second partial autocorrelation coefficient measures the correlation between $x_{t}$ and $x_{t-2}$ taking into account the effect of $x_{t-1}$. Similarly, the fourth partial autocorrelation coefficient is a measure of the correlation between $x_{t}$ and $x_{t-4}$ taking into account the effects of $x_{t-1}$, $x_{t-2}$, and $x_{t-3}$.
The R code below provides a demonstration.
Lastly, the ACF and PACF are important time-domain tools to help one understand the time-series properties of the data. They are used extensively in the Box-Jenkins methodology for identifying ARIMA models.
    # Simulate an AR(2) process ---

    x <- c(rep(0,200))    # Vector to hold the simulated series
    w <- rnorm(200)       # Errors drawn from normal distribution
    phi <- c(1.58,-0.64)  # Autoregressive paramters 

    # Simulation
    for(t in 3:200) x[t] <- phi[1] * x[t-1] + phi[2] * x[t-2] + w[t]

    # Housekeeping
    x <- ts(x[-(1:2)]) # Remove leading zeros
    maxLags <- 6       # Number of lagged series for regressions

    # Prepare data for lm() function
    data <- ts(embed(c(rep(NA,maxLags), x), maxLags+1)) # Dataset 
    colnames(data) <- c("x",paste0("AR",1:6))           # Varnames

    # Autocorrelation Function ---
    lm(x ~ AR1, data=data)
    acf(x)$acf[2]

    lm(x ~ AR2, data=data)
    acf(x)$acf[3]

    lm(x ~ AR3, data=data)
    acf(x)$acf[4]

    lm(x ~ AR4, data=data)
    acf(x)$acf[5]

    lm(x ~ AR5, data=data)
    acf(x)$acf[6]

    # Partial Autocorrelation Function ---
    lm(x ~ AR1, data=data)
    pacf(x)$acf[1]

    lm(x ~ AR1 + AR2, data=data)
    pacf(x)$acf[2]

    lm(x ~ AR1 + AR2 + AR3, data=data)
    pacf(x)$acf[3]

    lm(x ~ AR1 + AR2 + AR3 + AR4, data=data)
    pacf(x)$acf[4]

    lm(x ~ AR1 + AR2 + AR3 + AR4 + AR5, data=data)
    pacf(x)$acf[5]

    lm(x ~ AR1 + AR2 + AR3 + AR4 + AR5 + AR6, data=data)
    pacf(x)$acf[6]

There are small differences between some of the coefficients, but this is just related to the estimation methods used. The PACF can be estimated via linear regressions, but software typically use approximation methods such as the Yule-Walker equations and other methods of estimation.
