I used to assume that I knew what autocorrelation is and that acf
function in R produces what I understand as autocorrelation function, but I must be missing some detail because the results are not what I expect.
My understanding is that autocorrelation is correlation of a series with itself moved some lag away. For example, if x=1:10
we have autocorrelations:
> # lag 0
> cor(1:10, 1:10)
[1] 1
> # lag 1
> cor(1:9,2:10)
[1] 1
> # lag 2
> cor(1:8,3:10)
[1] 1
That is, for a linear series, autocorrelation should be 1 for any lag.
However, that's not the result from acf
function:
acf(1:10)
> acf(1:10)[0:9]
Autocorrelations of series ‘1:10’, by lag
0 1 2 3 4 5 6 7 8 9
1.000 0.700 0.412 0.148 -0.079 -0.258 -0.376 -0.421 -0.382 -0.245
Which are clearly different than the autocorrelation I computed by lagging the series.
My guess is that acf
function is somehow correcting for sample size, because the discrepancy is smaller when sample size doesn't change much (eg. by using 1:1000 instead of 1:10). In addition, I would expect the dashed blue line of significance in the plot to expand outwards as sample size decreases, but it is drawn as an horizontal line, and therefore as the threshold isn't expanding outwards it's reasonable to correct correlation inwards to keep constant the significance at threshold.
Then the questions are:
- Does
acf
correct for sample size? And if the answer is yes, how does it correct for sample size? - Does that correction the explain the difference between the correlation computed by lagging the series and the result of
acf
function? - If not, is my understanding of what autocorrelation is and what
acf
does wrong?