1
$\begingroup$

I have computed correlation coefficients for 90 day increments of a time series (i.e., one coefficient for days 0-90, 91-180, etc.). It was computationally very expensive to compute these and I would like to analyze correlations over 6 month periods, 1 year periods etc. Is it possible to some how to combine these coefficients? I.e., can I get the correlation over 180 days as 0.5*c1+0.5*c2?

I know in the general case this cannot be done, but because the coefficients are computed from the same number of samples, is it possible?

$\endgroup$
4
$\begingroup$

The sample correlation coefficient is

$$\sum (x_i-\bar{x})(y_i-\bar{y}) / \sqrt{\sum (x_i-\bar{x})^2 \sum(y_i-\bar{y})^2}$$

so the best thing would be to save $\sum(x-\bar{x})(y-\bar{y})$, $\sum(x-\bar{x})^2$ and $\sum(y-\bar{y})^2$ for the different windows. These could easily be combined, even with different sample sizes in the different windows.

If the denominators are approximately constant across 90 day increments, then you could just take the average of the correlations.

I'm surprised you say that this is computationally expensive. I wouldn't think it would take much more time than calculating the sample mean.

$\endgroup$
  • 2
    $\begingroup$ $\bar{x}$ and $\bar{y}$ may also change between the two samples. $\endgroup$ – Henry Aug 28 '11 at 13:47
  • $\begingroup$ yes, it seems like mathoverflow.net/a/57914 is closer to the solution. $\endgroup$ – alexey May 6 '16 at 20:41
1
$\begingroup$

To echo Karl, it's probably symptomatic of a larger problem if that was computationally expensive in any way. You can't get much better than linear time with stuff like this, and that should have been linear time. Perhaps the most productive thing would be to analyze you algorithms and make sure there's nothing obviously wrong there. We'd all be happy to help if you can post a snippet.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.