Context: I'm working on a machine learning problem where I'm using multivariate normal likelihood which requires calculating determinant and inverting the covariance matrix. I'm trying to generate some simulation data to prove that the method works but without success.
Question: Under what condition is a sample covariance matrix invertible? To illustrate, consider the below Python code which generates: 1) 3 random walks; 2) random walks around a trend. (1) returns determinant of zero and (2) returns a non-zero determinant. So what condition must be satisfied in order to generate a covariance matrix that's invertible?
import numpy as np # creates 3 random walks y = np.cumsum(np.random.normal(0., 0.1, (50, 3)), axis=0) # calculate covariance matrix np.cov(y, rowvar=False) c = np.cov(y, rowvar=False) # determinant is basically zero print(np.linalg.det(c)) # add a linear trend to the 3 random walks y = y + np.atleast_2d(np.arange(50)).T c = np.cov(y, rowvar=False) # prints 0.2218351316148853 np.linalg.det(c)