# Condition for covariance matrix to be non-invertible

Context: I'm working on a machine learning problem where I'm using multivariate normal likelihood which requires calculating determinant and inverting the covariance matrix. I'm trying to generate some simulation data to prove that the method works but without success.

Question: Under what condition is a sample covariance matrix invertible? To illustrate, consider the below Python code which generates: 1) 3 random walks; 2) random walks around a trend. (1) returns determinant of zero and (2) returns a non-zero determinant. So what condition must be satisfied in order to generate a covariance matrix that's invertible?

import numpy as np

# creates 3 random walks
y = np.cumsum(np.random.normal(0., 0.1, (50, 3)), axis=0)

# calculate covariance matrix
np.cov(y, rowvar=False)
c = np.cov(y, rowvar=False)

# determinant is basically zero
print(np.linalg.det(c))

# add a linear trend to the 3 random walks
y = y + np.atleast_2d(np.arange(50)).T
c = np.cov(y, rowvar=False)

# prints 0.2218351316148853
np.linalg.det(c)


[$$\mathbf{X^\top X}$$] can be singular whenever the data-generating distribution truly has no variance in some direction, or when no variance is observed in some direction because there are fewer examples (rows of $$\mathbf{X}$$) than input features (columns of $$\mathbf{X}$$).
Sometimes, the sample covariance matrix will be singular and you have to face it. There are approaches to handle such situations, as can be seen here, with $$\ell_1$$ regularization.
• Thanks. It turns out that the way I was estimating the covariance matrix meant that it will more often than not be non-invertible because, as you've pointed out, I don't have enough rows to estimate $XX^T$ (I was trying to estimate a $N \times N$ covariance matrix with just $N$ samples. Dec 17, 2021 at 3:53