# Sample covariance matrix

$\newcommand{\X}{\mathbf{X}} \renewcommand{\S}{\mathbf{S}} \newcommand{\I}{\mathbf{I}} \newcommand{\1}{\mathbf{1}}$ I found an article with an unusual (for me) covariance matrix.

Let $\X$ denote an $N\times T$ matrix of $T$ observations on a system of $N$ random variables representing $T$ returns on a universe of $N$ stocks. Then the sample covariance matrix is defined (p.\ 606) as:

$$\S= \frac{1}{T}\X\left( \I - \frac{1}{T} \1\1' \right) \X'$$

I find difficult to understand the above estimator. I would normally use something like:

$$\frac{1}{T-1} \left(\frac{1}{T}\X\I - \X\right) \left(\frac{1}{T}\X\I - \X\right)'$$

I believe there is a confusion of notation here. The paper uses $\I$ to denote the identity matrix, while you seem to use this symbol to denote a matrix where all its elements are equal to one, which the paper expresses using the notation $\1\1'$, $\1$ denoting a column vector of ones. (Moreover the paper defines $\X$ as a matrix where the series of a regressor is in one row, not in one column as is the most traditional case, but you don't seem troubled by this).

If we get past this notational issue, then the two expressions are the same (taking into account that you use $\I$ to denote $\1\1'$), since the matrix (using the paper's notation) $$Q_T = \I - \frac{1}{T} \1\1'$$ is idempotent, $Q_TQ_T= Q_T$ and symmetric, $Q_T' = Q_T$, and what it does is to de-mean the regressors (center them around their mean).

Assume we have 2 regressors $x,z$, with three observations. The authors define the regressor matrix as $2 \times 3$ $$X = \left [\begin{matrix} x_{1} & x_{2} & x_{3}\\ z_{1} & z_{2} & z_{3}\\ \end{matrix}\right]$$

and so $$Q_3 = \left[ \begin{matrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\\ \end{matrix} \right] - \frac{1}{T} \left[ \begin{matrix} 1 & 1 & 1\\ 1 & 1 & 1\\ 1 & 1 & 1\\ \end{matrix} \right]$$

Calculating $Q_3 X'$ shows the effect of $Q_3$.

Then the authors expression is compactly written

$$S = \frac 1 T XQ_TX' =\frac 1 T XQ_TQ_TX' = \frac 1 T(Q_T'X')'(Q_TX') = \frac 1 T(Q_TX')'(Q_TX')$$

where we have used the properties of the $Q_T$ matrix stated. If we define the transpose matrix of regressors de-meaned by $\tilde X' = Q_TX'$, then we have

$$S = \frac 1 T\tilde X \tilde X'$$

which has dimensions $2 \times 2$ as it should, and is the sample covariance matrix, given how the original regressor matrix is defined.

If you too, write $\I$ for the identity matrix, then your expression is simply mistaken, since it does not lead to subtraction of the sample mean from the observations.

• Hello, sorry but until know I didn't understand why they did not express the sample covariance matrix by the usual form. In addition, they used the usual form in their matab code. Does this unusual form is similar to the usual form? if yes, how? thanks in advance Jun 9 '15 at 12:17
• Furthermore why they assume this unusual estimator, how they got it Jun 9 '15 at 12:28
• @Christina There is nothing really unusual here, except maybe the fact that the place the regressors in rows rather than in columns (which is the norm in certain corners). I have expanded the answer. Jun 9 '15 at 20:26
• Thank you very much really!! but I still have two little questions :) first, if we assume that the series of regressors was in one column not in rows (which is the usual case), does the formulation (and your great explanation ) here remain the same? Second, in their matlab code, they used simply the matrix $S=XX'/T$. So the equation $S = \frac 1 T\tilde X \tilde X'$ is only used to explain the fact of why the sample covariance matrix becomes not invertible when the dimension exceeds the number of regressors? Jun 10 '15 at 9:26
• Or we can also use $S = \frac 1 T\tilde X \tilde X'$ instead of $S=XX'/T$ ? Jun 10 '15 at 9:31

The simplest way to understand the equation for S is by rearranging it as follows, using the expression for means vector $m=\frac{1}{T}X1$ given in the same paper:

$S=\frac{1}{T}\left(XX'-T\left(\frac{1}{T}X1\right)\left(1'X'\frac{1}{T}\right)\right)=\frac{1}{T}XX'-mm'$.

It should be obvious now how the equation for S corresponds to a usual covariance formula $E[xy]-E[x]E[y]$

The confusion is in somewhat unusual notation $1$ for conformant vector of ones: a $T\times 1$ vector of all 1s. So, the mean vector $m$ is simply a mean of each variable $m_i=\frac{\sum_{t=1}^T X_{it}\times 1}{T}$, i.e. a product of $N\times T$ matrix and $T\times 1$ vector is $N\times 1$ vector. Note, that $X1$ is the sum of each variable.

Now, $11'$ is the product of $T\times 1$ and $1\times T$ vectors, it's a $T\times T$ matrix of all 1s.

$I$ is a usual $T\times T$ identity matrix.