Skip to main content
edited tags
Link
Stephan Kolassa
  • 130.8k
  • 22
  • 264
  • 497
Source Link
a06e
  • 4.6k
  • 2
  • 24
  • 52

Online update of Pearson coefficient

Suppose I have an online stream of data points $x_i,y_i$, where $i=1,2,\dots$. I want to compute the Pearson correlation coefficient between the vectors $\vec x$ and $\vec y$.

But here is the catch. I receive the points one by one, and computing the correlation from scratch with each new point would be too slow (at some point I cannot even store all the points at once).

So let $\rho_N$ be the Pearson correlation up to the $N$'th data point. Is there a way to efficiently update this to $\rho_{N+1}$ when I receive the next data point? (Probably I have to store some additional intermediate quantities as I receive more points).