# sequential/recursive/online calculation of sample covariance matrix

I am solving the next exercise, but I have spent a lot of time and I can´t. For random vectors, $X_1,X_2,...\in\mathbb{R}^p$ The sample covariance with $\Sigma_1=0$ is given by: $$\hat\Sigma_n=\frac{1}{n-1}\sum_{i=1}^n(X_i-\hat{\mu_n})(X_i-\hat{\mu_n})^T$$ Prove that $\hat\Sigma_n=\frac{n-1}{n-2}\hat\Sigma_{n-1}+\frac{1}{n}\Delta_n\Delta_n^T$

Where $\Delta_n=X_n-\hat\mu_{n-1}$

I have the next part:

I start with the right side of the equality that I need to prove. i.e: $$=\frac{n-2}{n-1}(\frac{1}{n-2}\sum_{i=1}^{n-1}(X_i-\hat{\mu}_{n-1})(X_i-\hat{\mu}_{n-1})^T)+\frac{1}{n}\Delta_n\Delta_n^T+\frac{1}{n}\Delta_n\Delta_n^T\\ =\frac{1}{n-1}(\sum_{i=1}^{n-1}X_iX_i^T-\sum_{i=1}^{n-1}X_i\hat{\mu}_{n-1}^T-\hat{\mu}_{n-1}\sum_{i=1}^{n-1}X_i^T+\sum_{i=1}^{n-1}\hat{\mu}_{n-1}\hat{\mu}_{n-1}^T)+\frac{1}{n}\Delta_n\Delta_n^T$$ $$=\frac{1}{n-1}\sum_{i=1}^{n-1}X_iX_i^T-\hat{\mu}_{n-1}\hat{\mu}_{n-1}^T-\hat{\mu}_{n-1}\hat{\mu}_{n-1}^T+\hat{\mu}_{n-1}\hat{\mu}_{n-1}^T+\frac{1}{n}\Delta_n\Delta_n^T$$ In the previous line I used the fact that $\hat\mu_{n-1}=\frac{1}{n-1}\sum_{i=1}^{n-1}X_i$ then I conclude that $\sum_{i=1}^{n-1}X_i=(n-1)\hat\mu_{n-1}$ and with the common denominator it is simplified.

From here, I have replaced with the definition given of $\Delta_n$ and $\hat\mu_{n-1}=\frac{1}{n-1}\sum_{i=1}^{n-1}X_i$

But, I can´t conclude the exercise.

Additionally, I have a question related, At some stage of doing this proof I got something like that: $X_n\hat\mu_{n-1}^T$ and $\hat\mu_{n-1}X_n^T$, Since I am using vectors, this is a product of vectors and I got a matrix, but both are different or equal? according to me this is different because one is the transpose of other and is not equal, I did an example in R and I didn't get the same result, I mean I got that are different, Is ok my reasoning?

• By working it out first for one-dimensional vectors you will see how the algebra goes. The solution will be identical for higher-dimensional vectors; indeed, by using vector and matrix notation you won't have to change anything.
– whuber
Commented Oct 30, 2017 at 15:11

It's easy if you write $$\hat\Sigma_n= \frac{1}{n-1}\sum_{i=1}^nX_i X_i^T - \frac{n}{n-1}\hat{\mu}_n\hat{\mu}_n^T.$$ Split up the sum over $n$ elements into two parts. One will involve the first $n-1$ terms (which you can make look like the previous sample covariance), and the second will involve the most recent $n$th term. Finally, find a recursive formula for $\hat{\mu}_n$, and then plug that in.
Edit: The correct formula should be $$\hat{\Sigma}_n = \frac{n-2}{n-1}\hat{\Sigma}_{n-1} + \frac{1}{n}\Delta_n \Delta_n^T.$$ I think you flipped that first fraction accidentally.