1
$\begingroup$

I'm measuring a phenomenon over several days. On any particular day, the phenomenon is well correlated. I was wondering how I could pool the correlations over several days to get a single estimate of how well correlated the data are.

The last observation yesterday has nothing to do with the first observation today, so I don't think computing the correlation as if it were a time series is appropriate.

What is the recommended approach for analyzing correlations of this nature?

$\endgroup$
2
$\begingroup$

You give very little detail or context. It seems as you for multiple days have observations of some $(x,y)$ pairs that day, with independence between days. Neither you have told us what kind of correlation, I will assume now pearson correlation. If you have data for $T$ days, then, you will have estimated correlation coefficients $r_1, \dotsc, r_T$ and assume they are independent estimates of the same underlying true correlation $\rho$. So you want a common, or pooled, estimate, $\hat{\rho}$ summarizing the individual estimates.

Lets say you have observed $n$ pairs each day. The usual way of pooling estimates is to weight by the inverse variance. That is difficult in this case, since an expression for the variance is complicated and depends on the unknown $\rho$, see https://en.wikipedia.org/wiki/Pearson_correlation_coefficient or Expected value and variance of sample correlation But one idea could be to use this iteratively, starting with equal weights.

Another idea is to use the Fisher transform $F(r)=\frac12 \ln\left(\frac{1+r}{1-r}\right)$ which have approximately constant variance $\text{Var}(F(r))=\frac{1}{n-3}$. So you can take the average (equal weights) of $F(r_1), \dotsc, F(r_T)$ and the inverse transform of the result.

Googling for "pooling of correlation" gives a lot of relevant hits.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.