5
$\begingroup$

Assume we have N independent and identically distributed random vectors $X_1, X_2, ..., X_N$ where each of them is of size p $\times$ 1. The sample covariance matrix, denoted here by $S$, is computed as $(1/N)XX'$ where $X$ is now a matrix having the $X_i$ as columns, $i = 1, 2, ..., N$.

As we all know that $S$, has no structure, unbiased but has a lot of estimation error(ill-conditioned). In addition, $S$ becomes singular when $p>=N$.

So the obvious idea is to try by shrinking the unbiased covariance matrix $S$ towards a biased (with less variance) structured covariance often called "The shrinkage Target". The shrinkage formula can be expressed as follows:

$~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~$$(1-\alpha)S + \alpha F$

Where $\alpha$ is the shrinkage intensity (between 0 and 1), and $F$ is the shrinkage target. In shrinkage, the most challenging part is how to compute (automatically) the shrinkage intensity. So following the article of Ledoit and Wolf in 2003 "Improved Estimation of the Covariance Matrix of Stock Returns With an Application to Portfolio Selection", they computed an automatic optimal shrinkage intensity after minimizing a certain loss function based on the Frobenius distance.

Ok great! My question now is from this article. In page 8, I want to know how they just computed $[E(\alpha f_{ij} + (1 - \alpha)s_{ij} - \delta_{ij}]^2$. Ok, in the article, they used the single index model as shrinkage target $F$. But in fact, let us compute this squared expected value for a general shrinkage target. I just want to know how they computed this squared expectation (I am just interested for this expectation in page 8 before they did the derivation w.r.t $\alpha$)...

Kindly any help will be very very appreciated!!

| cite | improve this question | |
$\endgroup$
  • 1
    $\begingroup$ Maybe I'm looking at the wrong version, but on p.7 of ledoit.net/ole2.pdf they have $(\mathbb{E}[\alpha f_{ij} + (1-\alpha)s_{ij} - \sigma_{ij}])^2$ (not $\delta_{ij}^2$). If that's the right version, then they just wrote $$ \alpha (f-\phi) + (1-\alpha)(s-\sigma) + \alpha \phi + (1-\alpha)\sigma - \sigma, $$ where $\phi=\mathbb{E}[f]$ and $\sigma=\mathbb{E}[s]$ and expanded the expectation of the square as $$ \alpha^2 \mathbb{V}[f] + (1-\alpha)^2\mathbb{V}[s] + 2\alpha(1-\alpha)\mathrm{Cov}(f,s) + \alpha^2(\phi - \sigma)^2 $$ using the usual definition of variance. Is this close? $\endgroup$ – Kirill Aug 1 '15 at 2:41
  • $\begingroup$ Thank you very much for your time and reading my problem! but really I am still not understanding. Can you please put your explanation as an answer and explain me in details how you got your first line and then how you got $2 \alpha (1-\alpha)Cov(f,s) + \alpha^2 (\phi - \delta)^2$. And can you generalize your answer by considering that F is an identity matrix. Kindly your answer will be sooo appreciated for me! $\endgroup$ – Christina Aug 1 '15 at 10:23
  • $\begingroup$ Wow Christina, you really seem to have some kind of covariance estimation and shrinkage thing going on with many questions on this general topic. Are you doing some kind of research in the area? $\endgroup$ – Mark L. Stone Aug 1 '15 at 17:43
  • $\begingroup$ Hallo, yes I want to present next week at my university the work of this article in details and some alternatives of it such as the work of shafer and strimmer in 2005. This is a great topic really $\endgroup$ – Christina Aug 2 '15 at 11:09
1
$\begingroup$

What they evaluate there is $$ \mathbb{E}\big[ (\alpha f + (1-\alpha)s - \sigma)^2 \big], $$ (note this isn't identical to what you wrote in the question) where you are given $$ \mathbb{E}[f] = \phi, \qquad \mathbb{E}[s] = \sigma. $$

They evaluate this by writing $$ \alpha f + (1-\alpha)s-\sigma =: X, $$ and using the identity $$ \mathbb{E}[X^2] = \mathbb{V}[X] + (\mathbb{E}[X])^2. $$ Since $\phi$ and $\sigma$ are constants here, $$ \mathbb{V}[X] = \alpha^2 \mathbb{V}[f] + (1-\alpha)^2 \mathbb{V}[s] + \alpha(1-\alpha)\mathrm{Cov}[f,s], $$ and $$ \mathbb{E}[X] = \alpha(\phi-\sigma) $$ due to $\mathbb{E}[f]=\phi$, $\mathbb{E}[s]=\sigma$, and the usual variance-of-sum formula.

To generalize this to any other $\phi$, note that none of the properties of their matrices $\mathbf{F}$, $\mathbf{S}$, $\mathbf{\Sigma}$ are actually being used: this is simply an identity in terms of random variables $f,s$ and their means $\phi,\sigma$. It does not matter whether $\phi$ are the identity matrix elements or something else.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Okay so their proposal was very general and can be applied to any other target matrices as well. Thank you very much for your time. best regards $\endgroup$ – Christina Aug 2 '15 at 11:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.