Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

$\newcommand{\X}{\mathbf{X}} \renewcommand{\S}{\mathbf{S}} \newcommand{\I}{\mathbf{I}} \newcommand{\1}{\mathbf{1}} $ I found an article with an unusual (for me) covariance matrix.

Let $\X$ denote an $N\times T$ matrix of $T$ observations on a system of $N$ random variables representing $T$ returns on a universe of $N$ stocks. Then the sample covariance matrix is defined (p.\ 606) as:

$$ \S= \frac{1}{T}\X\left( \I - \frac{1}{T} \1\1' \right) \X' $$

I find difficult to understand the above estimator. I would normally use something like:

$$ \frac{1}{T-1} \left(\frac{1}{T}\X\I - \X\right) \left(\frac{1}{T}\X\I - \X\right)' $$

share|improve this question
Great question. Thanks – Christina Jun 8 '15 at 16:43
up vote 4 down vote accepted

I believe there is a confusion of notation here. The paper uses $\I$ to denote the identity matrix, while you seem to use this symbol to denote a matrix where all its elements are equal to one, which the paper expresses using the notation $\1\1'$, $\1$ denoting a column vector of ones. (Moreover the paper defines $\X$ as a matrix where the series of a regressor is in one row, not in one column as is the most traditional case, but you don't seem troubled by this).

If we get past this notational issue, then the two expressions are the same (taking into account that you use $\I$ to denote $\1\1'$), since the matrix (using the paper's notation) $$Q_T = \I - \frac{1}{T} \1\1'$$ is idempotent, $Q_TQ_T= Q_T$ and symmetric, $Q_T' = Q_T$, and what it does is to de-mean the regressors (center them around their mean).

Assume we have 2 regressors $x,z$, with three observations. The authors define the regressor matrix as $ 2 \times 3$ $$X = \left [\begin{matrix} x_{1} & x_{2} & x_{3}\\ z_{1} & z_{2} & z_{3}\\ \end{matrix}\right]$$

and so $$Q_3 = \left[ \begin{matrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\\ \end{matrix} \right] - \frac{1}{T} \left[ \begin{matrix} 1 & 1 & 1\\ 1 & 1 & 1\\ 1 & 1 & 1\\ \end{matrix} \right]$$

Calculating $Q_3 X'$ shows the effect of $Q_3$.

Then the authors expression is compactly written

$$S = \frac 1 T XQ_TX' =\frac 1 T XQ_TQ_TX' = \frac 1 T(Q_T'X')'(Q_TX') = \frac 1 T(Q_TX')'(Q_TX')$$

where we have used the properties of the $Q_T$ matrix stated. If we define the transpose matrix of regressors de-meaned by $\tilde X' = Q_TX'$, then we have

$$S = \frac 1 T\tilde X \tilde X'$$

which has dimensions $2 \times 2$ as it should, and is the sample covariance matrix, given how the original regressor matrix is defined.

If you too, write $\I$ for the identity matrix, then your expression is simply mistaken, since it does not lead to subtraction of the sample mean from the observations.

share|improve this answer
Hello, sorry but until know I didn't understand why they did not express the sample covariance matrix by the usual form. In addition, they used the usual form in their matab code. Does this unusual form is similar to the usual form? if yes, how? thanks in advance – Christina Jun 9 '15 at 12:17
Furthermore why they assume this unusual estimator, how they got it – Christina Jun 9 '15 at 12:28
@Christina There is nothing really unusual here, except maybe the fact that the place the regressors in rows rather than in columns (which is the norm in certain corners). I have expanded the answer. – Alecos Papadopoulos Jun 9 '15 at 20:26
Thank you very much really!! but I still have two little questions :) first, if we assume that the series of regressors was in one column not in rows (which is the usual case), does the formulation (and your great explanation ) here remain the same? Second, in their matlab code, they used simply the matrix $S=XX'/T$. So the equation $S = \frac 1 T\tilde X \tilde X'$ is only used to explain the fact of why the sample covariance matrix becomes not invertible when the dimension exceeds the number of regressors? – Christina Jun 10 '15 at 9:26
Or we can also use $S = \frac 1 T\tilde X \tilde X'$ instead of $S=XX'/T$ ? – Christina Jun 10 '15 at 9:31

The simplest way to understand the equation for S is by rearranging it as follows, using the expression for means vector $m=\frac{1}{T}X1$ given in the same paper:


It should be obvious now how the equation for S corresponds to a usual covariance formula $E[xy]-E[x]E[y]$

The confusion is in somewhat unusual notation $1$ for conformant vector of ones: a $T\times 1$ vector of all 1s. So, the mean vector $m$ is simply a mean of each variable $m_i=\frac{\sum_{t=1}^T X_{it}\times 1}{T}$, i.e. a product of $N\times T$ matrix and $T\times 1$ vector is $N\times 1$ vector. Note, that $X1$ is the sum of each variable.

Now, $11'$ is the product of $T\times 1$ and $1\times T$ vectors, it's a $T\times T$ matrix of all 1s.

$I$ is a usual $T\times T$ identity matrix.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.