Sample covariance matrix

Question

$\newcommand{\X}{\mathbf{X}} \renewcommand{\S}{\mathbf{S}} \newcommand{\I}{\mathbf{I}} \newcommand{\1}{\mathbf{1}} $ I found an article with an unusual (for me) covariance matrix.

Let $\X$ denote an $N\times T$ matrix of $T$ observations on a system of $N$ random variables representing $T$ returns on a universe of $N$ stocks. Then the sample covariance matrix is defined (p.\ 606) as:

$$ \S= \frac{1}{T}\X\left( \I - \frac{1}{T} \1\1' \right) \X' $$

I find difficult to understand the above estimator. I would normally use something like:

$$ \frac{1}{T-1} \left(\frac{1}{T}\X\I - \X\right) \left(\frac{1}{T}\X\I - \X\right)' $$

Answer 1

I believe there is a confusion of notation here. The paper uses $\mathbf I$ to denote the identity matrix, while you seem to use this symbol to denote a matrix where all its elements are equal to one, which the paper expresses using the notation $\mathbf1\mathbf1'$, $\mathbf1$ denoting a column vector of ones. (Moreover the paper defines $\mathbf X$ as a matrix where the series of a regressor is in one row, not in one column as is the most traditional case, but you don't seem troubled by this).

If we get past this notational issue, then the two expressions are the same (taking into account that you use $\mathbf I$ to denote $\mathbf 1\mathbf 1'$), since the matrix (using the paper's notation) $$Q_T = \mathbf I - \frac{1}{T} \mathbf 1\mathbf1'$$ is idempotent, $Q_TQ_T= Q_T$ and symmetric, $Q_T' = Q_T$, and what it does is to de-mean the regressors (center them around their mean).

Assume we have 2 regressors $x,z$, with three observations. The authors define the regressor matrix as $ 2 \times 3$ $$X = \left [\begin{matrix} x_{1} & x_{2} & x_{3}\\ z_{1} & z_{2} & z_{3}\\ \end{matrix}\right]$$

and so $$Q_3 = \left[ \begin{matrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\\ \end{matrix} \right] - \frac{1}{T} \left[ \begin{matrix} 1 & 1 & 1\\ 1 & 1 & 1\\ 1 & 1 & 1\\ \end{matrix} \right]$$

Calculating $Q_3 X'$ shows the effect of $Q_3$.

Then the authors expression is compactly written

$$S = \frac 1 T XQ_TX' =\frac 1 T XQ_TQ_TX' = \frac 1 T(Q_T'X')'(Q_TX') = \frac 1 T(Q_TX')'(Q_TX')$$

where we have used the properties of the $Q_T$ matrix stated. If we define the transpose matrix of regressors de-meaned by $\tilde X' = Q_TX'$, then we have

$$S = \frac 1 T\tilde X \tilde X'$$

which has dimensions $2 \times 2$ as it should, and is the sample covariance matrix, given how the original regressor matrix is defined.

If you too, write $\mathbf I$ for the identity matrix, then your expression is simply mistaken, since it does not lead to subtraction of the sample mean from the observations.

Answer 2

The simplest way to understand the equation for S is by rearranging it as follows, using the expression for means vector $m=\frac{1}{T}X1$ given in the same paper:

$S=\frac{1}{T}\left(XX'-T\left(\frac{1}{T}X1\right)\left(1'X'\frac{1}{T}\right)\right)=\frac{1}{T}XX'-mm'$.

It should be obvious now how the equation for S corresponds to a usual covariance formula $E[xy]-E[x]E[y]$

The confusion is in somewhat unusual notation $1$ for conformant vector of ones: a $T\times 1$ vector of all 1s. So, the mean vector $m$ is simply a mean of each variable $m_i=\frac{\sum_{t=1}^T X_{it}\times 1}{T}$, i.e. a product of $N\times T$ matrix and $T\times 1$ vector is $N\times 1$ vector. Note, that $X1$ is the sum of each variable.

Now, $11'$ is the product of $T\times 1$ and $1\times T$ vectors, it's a $T\times T$ matrix of all 1s.

$I$ is a usual $T\times T$ identity matrix.