# Differencing independent random variables makes autocorrelation appear

r<- diff(rnorm(100))
acf(r)


Why there is a negative auto correlation? What does it mean?

• Hi Daniele, I'm having trouble replicating your work. The acf() function in the stats:: package returns a plot (unless the argument plot = FALSE is supplied), not a single (positive or negative) value. What do you mean by "negative autocorrelation"? Do you mean to ask why one or more of the bars in the acf plot are negative? Dec 13 '17 at 23:36
• Ah yes yes, you have to do acf(r[2:100]) or you will get an error, however for lag=1 there is a negative auto correlation Dec 14 '17 at 7:48
• Which package is your acf() function from? Here are my results: > acf(r[2:100]) Error in na.fail.default(as.ts(x)) : missing values in object Dec 14 '17 at 13:43
• it is from Rstudio, I think it is stats, if it returns NA r[2:100] or r[1:99] Dec 14 '17 at 14:00
• There are only 99 values in r: the difference operator diff() will always have $n - 1$ values. Therefore, there isn't a 100th value of r, so r[100] will return an NA, and acf(r[2:100]) will error. Dec 14 '17 at 20:18

Let $x_t$ be the observed value of a stationary random process at index $t$ with mean and variance $\mu$ and $\sigma^2$, respectively. The autocorrelation of $x_t$ at lag $s$ is defined as

$$R(s - t) := \frac{E\left[(x_t - \mu)(x_s - \mu)\right]}{\sigma^2}.$$

Now let's go back to your toy example in R.

set.seed(8675309)
x_t <- rnorm(100)
r <- diff(x_t)
acf(r)


Yes, we do see some negative values in our autocorrelation plot. Let's double-check those calculations. The first spike (lag $s = 0)$ in the graph is the correlation between $r_t$ and $r_{t - 0}$ (i.e., itself); this correlation is $+1$ because any random variable is perfectly correlated with itself. The second spike is the correlation between $r_t$ and $r_{t - 1}$. This value is

> cor(r[2:99], r[1:98])
-0.4051325


which is what we see in our graph. Now about your question of why this autocorrelation value of the first lag is negative, we have to remember that we aren't plotting the ACF of a white noise variable $x_t$ anymore: we are plotting the ACF of the first difference $r_t$.

Remember that the difference operator can be a nice tool to deal with autocorrelation. Now let's look for some autocorrelation within our original $x_t$ using the ACF plot:

As we can see, there isn't any autocorrelation to be fixed in the first place, so the difference operator is actually creating an autocorrelation problem through overdifferencing. This previous answer gives an excellent walk through the mathematics behind this.

Because xt <- rnorm(100) generates $\{x_t\}_{t=1}^{100} \sim IID(0,1)$, and yt <- diff(xt) generates $$Y_t = X_t - X_{t-1}, \hspace{10mm} t=2,\ldots,100.$$ The theoretical autocorrelation function of $Y_t$ is \begin{align*} \gamma_Y(h) &= E[Y_{t+h}Y_t] = E[(X_{t+h} - X_{t+h-1})(X_t - X_{t-1})] \\ &= \gamma_X(h) - \gamma_X(h+1) - \gamma_X(h-1) + \gamma_X(h). \end{align*} $\gamma_X(h)$ is zero for $h > 0$, not $\gamma_Y(h)$. So it's not surprise that $\hat{\gamma}_Y(h)$, calculated with acf doesn't look like this either.
This is a first sign of overdifferencing in practical work. This is why it happens, suppose that you have a stationary series, or even stronger, the i.i.d. series with variance $\sigma^2$: $$x_t\sim f(0,\sigma^2)$$ Look at what happens to the differenced series: $$\Delta x_t=x_t-x_{t-1}$$ The variance is $Var[\Delta x_t]=E[x_t^2+x_{t-1}^2-2x_tx_{t-1}]=2\sigma^2$
The numerator of autocorrelation function (ACF) is: $$E[\Delta x_t\times \Delta x_{t-1}]=E[x_tx_{t-1}-x_{t-1}^2+x_tx_{t-2}-x_{t-1}x_{t-2}]=-E[x_t^2]=-\sigma^2$$ So your acf is: $$\rho(1)=\frac{-\sigma^2}{2\sigma^2}=-1/2$$ It's easy to show that because of non-overlapping terms in $\Delta x_t,\Delta x_{t-1}$ ACF cuts off after 1 lag, i.e. $$\rho(L>1)=0$$ That brings you a very distinctive feature of the once over differenced series: strong neg correlation in lag 1, then no correlation for higher lags.