Condition for covariance matrix to be non-invertible

Question

Context: I'm working on a machine learning problem where I'm using multivariate normal likelihood which requires calculating determinant and inverting the covariance matrix. I'm trying to generate some simulation data to prove that the method works but without success.

Question: Under what condition is a sample covariance matrix invertible? To illustrate, consider the below Python code which generates: 1) 3 random walks; 2) random walks around a trend. (1) returns determinant of zero and (2) returns a non-zero determinant. So what condition must be satisfied in order to generate a covariance matrix that's invertible?

import numpy as np

# creates 3 random walks
y = np.cumsum(np.random.normal(0., 0.1, (50, 3)), axis=0)

# calculate covariance matrix
np.cov(y, rowvar=False)
c = np.cov(y, rowvar=False)

# determinant is basically zero
print(np.linalg.det(c))


# add a linear trend to the 3 random walks
y = y + np.atleast_2d(np.arange(50)).T
c = np.cov(y, rowvar=False)

# prints 0.2218351316148853
np.linalg.det(c)

Answer 1

In Deep Learning by Goodfellow et al, the authors write on page 232:

[$\mathbf{X^\top X}$] can be singular whenever the data-generating distribution truly has no variance in some direction, or when no variance is observed in some direction because there are fewer examples (rows of $\mathbf{X}$) than input features (columns of $\mathbf{X}$).

This questiton/answer, from which the previous paragraph was taken, can be useful.

Sometimes, the sample covariance matrix will be singular and you have to face it. There are approaches to handle such situations, as can be seen here, with $\ell_1$ regularization.