As we all know that the sample covariance matrix $(S = (s_{ij}))$ is postive definite when the number of observations is smaller than the number of samples, that is n>p. But, the sample covariance matrix is always badely conditioned. In other words, few of its eigenvalues are large and the rest are too small especially when p>>n.

That is why, a very famous regularization is usually used and it is called "Shrinkage". Ledoit and Wolf (2004) used the Identity matrix as shrinkage target $T$ since it is always positive definite, well structured and very well conditioned (condition number = 1).

In the article of Schafer and Strimmer (click here to open the article), they were also interested to use a diagonal matrix as shrinkage target $T$ but now it is the "unequal variance, zero covariances". That is the target D:

$~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~$$t_{ij} = s_{ij}$ if $i=j$ and $0$ if $i$ $!=$ $j$.

In fact, as we know that the matrix $S$ is badely conditioned, so why they used its diagonal as target? I know that they used this target because it is always positive definite and it is a compromise between the targets A,B, C and E, F (as mentioned in their article, page 12), and unlike to the identity target, it shrinks only the covariances (so it does not shrink the variances).

1) Does their target (Target D) is well conditioned?

2) What will be better, shrinking both covariances and variances (as in the case of the identity target) or only the covariances (as in the case of Target D)?

3) If the eigenvalues of the matrix $S$ are spread out, does the eigenvalues of the diagonal of $S$ are not spread out? So if they are also spread out, in this case why they were interested only in Target D?

Any help will be very appreciated!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.