Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Why differencing a normal, i.i.d. time series $X$, generates a negatively correlated series $Y$?

> Acf(rnorm(5000, 0, 40))
> Acf(diff(rnorm(5000, 0, 40)))

enter image description here

I stumbled on this problem while looking at a time-series that I wanted to predict using an ARIMA model. The original series appeared to be non-stationary by looking at the plot, so I decided to apply a difference at lag 1 and check the acf/pacf. This seemed to indicate an MA(1) model but obviously there is something I'm missing.

enter image description here

share|improve this question
1  
Hint: $Y_n = X_n - X_{n-1}$ so $Y_n$ has zero mean. So, what's $\mathbb E Y_n Y_{n-1}$? Initial conclusion? Now check the autocorrelation at further lags. Final conclusion? –  cardinal May 11 '12 at 13:06
    
$EY_n|Y_{n-1}$ should be something like $-Y_{n-1}$, if $Y_n$ has 0 mean? But then so is $X_n$. It has 0 mean. I still don't see it. –  Robert Kubrick May 11 '12 at 13:30
1  
Robert: $\mathbb E Y_n Y_{n-1} = \mathbb E(X_n - X_{n-1})(X_{n-1}-X_{n-2})$. Can you continue by using the stated properties of $X_n$? –  cardinal May 11 '12 at 13:56
2  
@cardinal I appreciate your interest in my question, but this is not a classroom, or even a chat room. If you know the answer, please post an explanation and I will work from there to understand the properties. –  Robert Kubrick May 11 '12 at 14:29
2  
+1 - these comments are not meant for a discussion (however well-intentioned) designed to help the OP find the answer for themselves. CV isn't a site for teaching - if a simple hint isn't sufficient either leave it at that or write up an Answer. –  Gavin Simpson May 11 '12 at 14:51
show 6 more comments

2 Answers

up vote 6 down vote accepted

If I understand your comments correctly, you've overdifferenced, which is talked about in various guides.

EDIT: Your original series of numbers (rnorm(5000, 0, 40)) has, by definition and design, no relationship between adjacent numbers or every 2nd number or every 3rd number. It's "random" (pseudo-random, but not distinguishable from truly random by us mere mortals). So the ACF you calculate is random garbage.

But differencing takes that series of numbers and creates a new series which is related in a particular, deterministic way: subtraction of adjacent values. Consider your initial random number series: $(n_1, n_2, n_3, ...)$, then difference it to get $(d_1, d_2, ...)$. Both $d_1$ and $d_2$ are calculated using $n_2$, so you've now introduced autocorrelation at lag 1.

Now look at what happens at that lag 1. $n_2$ is used to calculate $d_1$ and $d_2$, once subtracting from and once being subtracted from. [Begin I'm-way-in-over-my-head part.] In order for $d_1$ and $d_2$ to have the same sign, we'd need to have $n_1 < n_2$ and $n_2 < n_3$ (or vice versa), which is less likely than the alternatives, so we expect that the autocorrelation will be negative. [End I'm-way-in-over-my-head part, gasping for air.]

share|improve this answer
    
Thanks, very clear explanation. By the way, I checked Box and Jennings 2008, there is no note about overdifferencing in this context. There are a few references to other papers/books in the appendix but I'm not sure it's even related to this case. –  Robert Kubrick May 11 '12 at 21:53
    
I nice online guide that mentions overdifferencing is: people.duke.edu/~rnau/411arim2.htm –  Wayne Dec 14 '13 at 18:56
add comment

It is a little late ..... but , review the Slutsky Effect where a linear (weighted ) combinations of i.i.d. values leads to a series with auto-correlative structure. This is why assuming any filter picket out of the blue can be dangerous. X11-ARIMA assumes a 16 period equally weighted average ( you can change 16 to another integer ) to smooth the series not knowing the impact of assumed filters. Long live analytics !

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.