Skip to main content
Source Link
Foobar
  • 369
  • 2
  • 9

What is the intuition behind the single-pass algorithm (Welford's method) for the corrected sum of squares?

The corrected sum of squares is the sum of squares of the deviations of a set of values about its mean.

$$ S = \sum_{i=1}^k\space\space(x_i - \bar x)^2 $$

We can calculate the mean in a streaming fashion, as follows:

$$ m_n = \frac{n-1}{n}m_{(n-1)} + \frac{1}{n}x_n $$

I understand the intuition behind this: the sum of the previous $n-1$ values was divided by $n-1$, so by multiplying those values by $\frac{n-1}{n}$, we down-weight the sum properly.

However, we can also calculate the full corrected sum of squares as follows:

$$ S_n = S_{n-1} + \frac{n-1}{n}\left( x_n - m_{n-1}\right)^2 $$

However, I don't have a good intuition for why this works. It looks like we use the previous corrected sum of squares value, and then add the square of the current value's deviation from the mean of all the previous values.

But, this algorithm doesn't make sense to me, even if it was derived logically.

These formulas are from "Note on a Method for Calculating Corrected Sums of Squares and Products" by B.P Welford.