1
$\begingroup$

Say I have a sample where $ N < p $, where $ N $ denotes the number of observations and $ p $ the number of variables. I know that the rank of the covariance matrix is then at most $ N $. I know that it is not positive definite since it is not invertible. However, I read from Wikipedia that a covariance matrix has to be at least positive semidefinite. Can I deduce positive semidefiniteness from here?

$\endgroup$

1 Answer 1

3
$\begingroup$

Yes, the sample covariance matrix will still be positive semi-definite.

To see this, note that if $X\in \mathbb{R}^{N\times p}$ is the data matrix (with observations in the rows and variables in the columns), then the sample covariance matrix is $C := \frac{1}{N-1}Y^TY\in \mathbb{R}^{p\times p}$, where $Y\in \mathbb{R}^{N\times p}$ is the matrix $X$ with each column's mean subtracted from that column's entries.

Note that $C$ is symmetric as $C^T = \left( \frac{1}{N-1}Y^TY\right)^T = \frac{1}{N-1}Y^TY=C$.

Also, for any $\mathbf{v}\in\mathbb{R}^{p}$, we have

$$\begin{align*} \mathbf{v}^T C \mathbf{v}&= \frac{1}{N-1}\mathbf{v}^TY^TY\mathbf{v}\\ &= \frac{1}{N-1} \left( Y\mathbf{v}\right)^T Y\mathbf{v}\\ &= \frac{1}{N-1}\left\| Y\mathbf{v}\right\|^{2}\\ &\ge 0. \end{align*} $$

Thus the sample covariance matrix $C$ is positive semi-definite.

$\endgroup$
3
  • $\begingroup$ So my question stems from an observation I had while working with a dataset. It considered the portfolio problem. I had some allocation for which the variance of the portfolio were negative. Indicating that the sample covariance matrix is rather indefinite. $\endgroup$ Nov 9, 2021 at 9:01
  • $\begingroup$ That should not be possible in theory. I would try to double check the covariance matrix and check if it is positive semi-definite (which you could do by verifying that it is symmetric and its eigenvalues are all non-negative). Otherwise perhaps numerical or rounding errors are causing this. $\endgroup$ Nov 9, 2021 at 9:05
  • $\begingroup$ You were right. The covariance matrix had some NAs due to non-overlapping values. These values were imputed. I assume that this is the reason for the negative eigenvalues. $\endgroup$ Nov 10, 2021 at 10:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.