Is every correlation matrix positive definite?

Question

I'm talking here about matrices of Pearson correlations.

I've often heard it said that all correlation matrices must be positive semidefinite. My understanding is that positive definite matrices must have eigenvalues $> 0$, while positive semidefinite matrices must have eigenvalues $\ge 0$. This makes me think that my question can be rephrased as "Is it possible for correlation matrices to have an eigenvalue $= 0$?"

Is it possible for a correlation matrix (generated from empirical data, with no missing data) to have an eigenvalue $= 0$, or an eigenvalue $< 0$? What if it was a population correlation matrix instead?

I read at the top answer to this question about covariance matrices that

Consider three variables, $X$, $Y$ and $Z = X+Y$. Their covariance matrix, $M$, is not positive definite, since there's a vector $z$ ($= (1, 1, -1)'$) for which $z'Mz$ is not positive.

However, if instead of a covariance matrix I do those calculations on a correlation matrix then $z'Mz$ comes out as positive. Thus I think that maybe the situation is different for correlation and covariance matrices.

My reason for asking is that I got asked over on stackoverflow, in relation to a question I asked there.

Answer 1

Correlation matrices need not be positive definite.

Consider a scalar random variable X having non-zero variance. Then the correlation matrix of X with itself is the matrix of all ones, which is positive semi-definite, but not positive definite.

As for sample correlation, consider sample data for the above, having first observation 1 and 1, and second observation 2 and 2. This results in sample correlation being the matrix of all ones, so not positive definite.

A sample correlation matrix, if computed in exact arithmetic (i.e., with no roundoff error) can not have negative eigenvalues.

Answer 2

The answers by @yoki and @MarkLStone (+1 to both) both point out that a population correlation matrix can have zero eigenvalues if variables are linearly related (such as e.g. $X_1 = X_2$ in the example of @MarkLStone and $X_1 = 2X_2$ in the example of @yoki).

In addition to that, a sample correlation matrix will necessarily have zero eigenvalues if $n<p$, i.e. if the sample size is smaller than the number of variables. In this case covariance and correlation matrices will both be at most of rank $n-1$, so there will be at least $p-n+1$ zero eigenvalues. See Why is a sample covariance matrix singular when sample size is less than number of variables? and Why is the rank of covariance matrix at most $n-1$?

Answer 3

Consider $X$ to be an r.v. with mean 0 and variance of 1. Let $Y=2X$, and calculate the covariance matrix of $(X,Y)$. Since $2X=Y$, $E[Y^2]=4E[X^2]=\sigma_Y^2$, and $E[XY]=2E[X^2]$. Due to the zero mean configuration, the second moments are equal to the suitable covariances, for instance: $\mbox{Cov}(X,Y)=E[XY]-EXEY=E[XY]$.

So the covariance matrix will be: $$ \Lambda = \left(\array{1 & 2 \\ 2 & 4 }\right),$$ having a zero eigenvalue. The correlation matrix will be: $$ \Lambda = \left(\array{1 & 1 \\ 1 & 1 }\right),$$ having a zero eigenvalue as well. Due to the linear correspondence between $X$ and $Y$ it is easy to see why we get this correlation matrix - the diagonal will always be 1, and the off-diagonal is 1 because of the linear relationship.