As stated in this question, the maximum rank of covariance matrix is $n-1$ where $n$ is sample size and so if the dimension of covariance matrix is equal to the sample size, it would be singular. I can't understand why we subtract $1$ from the maximum rank $n$ of covariance matrix.

  • 1
    $\begingroup$ To get the intuition, think about $n=2$ points in 3D. What is the dimensionality of the subspace that these points lie in? Can you fit them on a line (1D subspace)? Or do you need a plane (2D subspace)? $\endgroup$
    – amoeba
    Oct 25, 2015 at 17:41
  • $\begingroup$ So you do understand that $n=2$ leads to rank 1 covariance matrix? Okay, let's take $n=3$ points. Can you see that you can always fit them on a 2D plane? $\endgroup$
    – amoeba
    Oct 25, 2015 at 17:47
  • 5
    $\begingroup$ @amoeba your example was clear but I can't understand what is the relationship between fitting hyper-plane in your example and covariance matrix? $\endgroup$ Oct 25, 2015 at 18:14
  • $\begingroup$ Sorry for delay ;) $\endgroup$ Oct 16, 2016 at 19:45

3 Answers 3


The unbiased estimator of the sample covariance matrix given $n$ data points $\newcommand{\x}{\mathbf x}\x_i \in \mathbb R^d$ is $$\mathbf C = \frac{1}{n-1}\sum_{i=1}^n (\x_i - \bar \x)(\x_i - \bar \x)^\top,$$ where $\bar \x = \sum \x_i /n$ is the average over all points. Let us denote $(\x_i-\bar \x)$ as $\newcommand{\z}{\mathbf z}\z_i$. The $\frac{1}{n-1}$ factor does not change the rank, and each term in the sum has (by definition) rank $1$, so the core of the question is as follows:

Why does $\sum \z_i\z_i^\top$ have rank $n-1$ and not rank $n$, as it would seem because we are summing $n$ rank-$1$ matrices?

The answer is that it happens because $\z_i$ are not independent. By construction, $\sum\z_i = 0$. So if you know $n-1$ of $\z_i$, then the last remaining $\z_n$ is completely determined; we are not summing $n$ independent rank-$1$ matrices, we are summing only $n-1$ independent rank-$1$ matrices and then adding one more rank-$1$ matrix that is fully linearly determined by the rest. This last addition does not change the overall rank.

We can see this directly if we rewrite $\sum\z_i = 0$ as $$\z_n = -\sum_{i=1}^{n-1}\z_i,$$ and now plug it into the above expression: $$\sum_{i=1}^n \z_i\z_i^\top = \sum_{i=1}^{n-1} \z_i\z_i^\top + \Big(-\sum_{i=1}^{n-1}\z_i\Big)\z_n^\top=\sum_{i=1}^{n-1} \z_i(\z_i-\z_n)^\top.$$ Now there is only $n-1$ terms left in the sum and it becomes clear that the whole sum can have at most rank $n-1$.

This result, by the way, hints to why the factor in the unbiased estimator of covariance is $\frac{1}{n-1}$ and not $\frac{1}{n}$.

The geometric intuition that I alluded to in the comments above is that one can always fit a 1D line to any two points in 2D and one can always fit a 2D plane to any three points in 3D, i.e. the dimensionality of the subspace is always $n-1$; this only works because we assume that this line (and plane) can be "moved around" in order to fit our points. "Positioning" this line (or plane) such that it passes through $\bar \x$ is equivalent of centering in the algebraic argument above.


A bit shorter, I believe, explanation goes like this:

Let us define matrix $n$ x $m$ matrix $x$ of sample data points where $n$ is a number of variables and $m$ is a number of samples for each variable. Let us assume that none of the variables are linearly dependent.

The rank of $x$ is $\min(n,m)$.

Let us define matrix $n$ x $m$ matrix $z$ of row-wise centered variables:

$z = x - E[x]$.

The rank of centered data becomes $\min(n,m-1)$, because each data row is now subjected to constraint:

$ \sum_{i=1}^{m}z_{*i} =0$.

It basically means we can recreate the entire $z$ matrix even if one of columns is removed.

The equation for sample covariance of $x$ becomes:

$ cov(x,x) = \frac{1}{m-1}zz^T $

Clearly, the rank of covariance matrix is the $rank(zz^T)$.

By rank-nullity theorem: $rank(zz^T) = rank(z) = \min(n,m-1)$.


For the same reason that you need at least two observations to estimate the variance of a single random variable.

  • 1
    $\begingroup$ But you don't need two! It depends on what you assume about that variable; and even then, you can always estimate the variance based on one value. There's going to be an issue of how good the estimate might be, but that's not really germane. $\endgroup$
    – whuber
    Mar 31, 2022 at 21:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.