1
$\begingroup$

Assume we have the sample covariance matrix $S_1 = XX'/k$ which is not positive definite (in fact it is positive semi-definite) and not well conditioned in very large dimension (large $p$, small $k$).

If someone applies a specific method to the inverse of $S_1$ (that is, the inverse of the sample covariance matrix: $S_1^{-1}$) and get an inverse but which is now positive definite and well conditioned matrix. If we re-inverse the resulting matrix (the estimated inverse), do we get a positive definite and well conditioned matrix? In other words, the new sample covariance matrix is now guaranteed to be + definite and well conditioned?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

To expand on Brian Borchers' answer:

Per sections 3.2 and 6.5 of "Nonlinear Shrinkage Estimation of Large-Dimensional Covariance Matrices", by Olivier Ledoit and Michael Wolf in The Annals of Statistics 2012, inverting (adjusted to be better conditioned) estimate of covariance may not yield the best estimate of its inverse, referred to as the Precision matrix, and vice versa. So, the inverse of the well-conditioned covariance matrix will be well-conditioned, but might not be the "optimal" estimate.

If X were the MLE (maximum likelihood estimate) of covariance matrix C, then by functional invariance of maximum likelihood estimation https://en.wikipedia.org/wiki/Maximum_likelihood , inv(X) would be the MLE of inv(C). In this case, X = sample covariance matrix, so inv(sample covariance matrix) is the MLE of inv(C). However, the adjustments to the MLE in order to get a better conditioned solution mean that the adjusted estimate is not the MLE of C, and therefore there is no a priori reason to believe that the inverse of the adjusted estimate of C will be the "best" estimate of inv(C). And that is the point of those sections in the referenced paper, at least with regard to the proposed estimator of C and what that means for the inverse of C.

Improve this answer
$\endgroup$
5
$\begingroup$

It's a fact of linear algebra that the inverse of a symmetric and positive definite matrix is also symmetric and positive definite and has the same condition number as the matrix you started with.

You haven't said what is actually being done to the sample covariance, and I have my doubts as to whether it is reasonable, however...

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you for your answer, but what do you mean about ''whether it is reasonable''. Do you mean that we can not actually make the inverse of the sample covariance +definite and well conditioned? $\endgroup$
    – Christina
    Commented Jun 10, 2015 at 20:28
  • 2
    $\begingroup$ You haven't said what you're doing to the inverse of the sample covariance matrix- there are lots of arbitrary ways to make it positive definite and well conditioned (e.g. by adding a large multiple of the identity matrix) but statistically justifying such an operation is a very different issue. $\endgroup$
    – Brian Borchers
    Commented Jun 10, 2015 at 21:02
  • $\begingroup$ Ok what if we assume that we render its inverse + definite and well-conditioned by using a specific method (I will read about these methods but it is just an assumption now). Do you know if in this case the estimation error of the sample covariance and of its new inverse are positively correlated or negatively correlated? thank you so much for your time :) $\endgroup$
    – Christina
    Commented Jun 10, 2015 at 21:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.