1
$\begingroup$

I am a developer and I am trying to compute the autocorrelation coefficient of lag 1 incrementally. The problem is I have to test the results with some certified results from NiST Datasets. The autocorrelation coefficient used in these datasets is the following (it uses the time series analysis definition of autocorrelation, as explained here):
$$r1 = \frac{\sum_2^N(y_{i} - \overline{y})(y_{i-1} - \overline{y})}{\sum_1^N(y_{i} - \overline{y})^2}$$

I have tried to reuse the covariance incremental formula and adapt it but it does not work. I got closest to the exact results and my results differ in the third decimal digit. It's not a problem of loss of significance since I am using BigDecimal and numerically-stable algorithms to compute the mean and variance.

I am using $$r1 = \frac{\sum_2^N(y_{i} - \overline{y_{1}})(y_{i-1} - \overline{y_{2}})}{\sum_1^N(y_{i} - \overline{y})^2}$$ which is the standard comoment divided by the second central moment. The problem here is I am using different means in computing them and I have to use the same mean in both terms. Nonetheless, I cannot adapt this to my current scheme. Currently, I process data incrementally. I have a class called Correlation, this class updates the estimators while receiving a new pair of values by updating two internal classes (Sample, which contains descriptive statistics of certain set of data).

I am updating the class Correlation with $$(y_{i-1}, y_{i})$$Then, the first value updates the first sample and the second one the second sample. I cannot keep the same mean in both samples. At least, I don't find any way of doing it. Any clue?

What I am looking for is an incremental way of computing the autocorrelation coefficient in one pass while mantaining my current scheme, if possible.

$\endgroup$
3
  • $\begingroup$ Which formula are you using now (that didn't work)? Can you explain the algorithm as you currently implement it? Have you tried it (against the above definition) on very small samples, such as n=3 or n=4, where differences should be highlighted more clearly? $\endgroup$
    – Glen_b
    Mar 5, 2015 at 21:52
  • $\begingroup$ I have edited the question in order to answer your questions. @Glen_b $\endgroup$
    – Jorge
    Mar 6, 2015 at 8:30
  • $\begingroup$ You might wanna have a look at Welford's technique: amstat.tandfonline.com/doi/abs/10.1080/00401706.1962.10490022 $\endgroup$
    – Amruth
    Sep 9, 2018 at 17:49

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.