1
$\begingroup$

If I have a sample of size n and the sample mean Xbar, sample covariance matrix S and inverse of covariance matrix, S^-1. How do I update the formula for each to add a new observation, say, X(n+1)? Sample mean and S should be easy. But I'm wondering how to update the formula for S^-1 using just these...

$\endgroup$
  • $\begingroup$ If it is feasible to work with Cholesky factorizations, it seems that Cholesky rank-1 updates would work here. Also, IIRC the Sherman-Morrison-Woodbury update mentioned below is not numerically stable. $\endgroup$ – chainD Feb 11 '18 at 7:54
3
$\begingroup$

Perhaps you are looking for the Sherman-Morrison-Woodbury update, which allows to perform a rank k update directly to the inverse of a matrix rather than inverting a rank k update to the matrix itself.

It is used heavily in statistics and optimization literature. The form is slightly daunting but it is easy to verify its correctness and it is well worth understanding as it is quite commonly used.

$\endgroup$
  • $\begingroup$ Thanks...that seems fine and I'll look through the proof too. However, I was trying to prove the formula for S(n+1), but I'm getting some extra term that doesn't cancel out. Is there a formal proof somewhere for sample covariance too? $\endgroup$ – Name LeftBlank Feb 11 '18 at 3:08
  • $\begingroup$ Perhaps this is what you are looking for? I remember deriving it by hand and it is easy to mess up the algebra. en.wikipedia.org/wiki/… $\endgroup$ – David Kozak Feb 11 '18 at 3:22
  • $\begingroup$ Yes that's where I saw the formula, however i'm getting an extra multiplication for the 2nd term, i.e. (n^2+1)/(n*(n+1)) times the second term. Ideally it should have canceled out, but I have checked my derivation a few times. Not sure what I'm missing. $\endgroup$ – Name LeftBlank Feb 11 '18 at 3:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.