# Positive definiteness of sample covariance matrix when $N < p$

Say I have a sample where $$N < p$$, where $$N$$ denotes the number of observations and $$p$$ the number of variables. I know that the rank of the covariance matrix is then at most $$N$$. I know that it is not positive definite since it is not invertible. However, I read from Wikipedia that a covariance matrix has to be at least positive semidefinite. Can I deduce positive semidefiniteness from here?

Yes, the sample covariance matrix will still be positive semi-definite.

To see this, note that if $$X\in \mathbb{R}^{N\times p}$$ is the data matrix (with observations in the rows and variables in the columns), then the sample covariance matrix is $$C := \frac{1}{N-1}Y^TY\in \mathbb{R}^{p\times p}$$, where $$Y\in \mathbb{R}^{N\times p}$$ is the matrix $$X$$ with each column's mean subtracted from that column's entries.

Note that $$C$$ is symmetric as $$C^T = \left( \frac{1}{N-1}Y^TY\right)^T = \frac{1}{N-1}Y^TY=C$$.

Also, for any $$\mathbf{v}\in\mathbb{R}^{p}$$, we have

\begin{align*} \mathbf{v}^T C \mathbf{v}&= \frac{1}{N-1}\mathbf{v}^TY^TY\mathbf{v}\\ &= \frac{1}{N-1} \left( Y\mathbf{v}\right)^T Y\mathbf{v}\\ &= \frac{1}{N-1}\left\| Y\mathbf{v}\right\|^{2}\\ &\ge 0. \end{align*}

Thus the sample covariance matrix $$C$$ is positive semi-definite.

• So my question stems from an observation I had while working with a dataset. It considered the portfolio problem. I had some allocation for which the variance of the portfolio were negative. Indicating that the sample covariance matrix is rather indefinite. Commented Nov 9, 2021 at 9:01
• That should not be possible in theory. I would try to double check the covariance matrix and check if it is positive semi-definite (which you could do by verifying that it is symmetric and its eigenvalues are all non-negative). Otherwise perhaps numerical or rounding errors are causing this. Commented Nov 9, 2021 at 9:05
• You were right. The covariance matrix had some NAs due to non-overlapping values. These values were imputed. I assume that this is the reason for the negative eigenvalues. Commented Nov 10, 2021 at 10:05