# Does lack of data affect covariance matrix estimate?

I am building some experiments using the multivariate normal probability density function to estimate the likelihood of a given sample to come from a distribution. For that, the PDF is built using as parameter the covariance matrix of data.

In my experiment, i am presencing an error when calculating the PDF, it warns me that i'm trying to do a division by zero. Looking at the formula of the MVPDF, the only parameter that can be zero in the denominator is the determinant of the covariance matrix. My hypothesis is that the correlation matrix is singular and its determinant is 0.

I'd like to know if this has any relation with the size of my dataset, and (if so) how can i workaround this problem.

Yes, but...

The sample correlation matrix is usually extremely noisy. There's this whole area of statistics on "cleaning" correlation matrices, e.g. see RMT approach here or a shrinkage method.

However, random matrices rarely have determinant equal zero unless you construct them in certain ways. So, you might be having some other issue on top of the noise. I'd take a closer look and try to diagnose the exact cause. Otherwise, you can always apply the above shrinkage to shut the problem with zero determinant, as long as you know it's not the red herring

• Re "rarely have determinant zero:" This might be an exaggeration, especially in some domains where such data occur all the time, such as chemical compositions (which, since they are known to sum to 100%, may be standardized to do so) as well as any other compositional data, such as exhaustive summaries of responses to multiple-choice questions.
– whuber
Commented Mar 31, 2021 at 19:29
• Thanks for the great suggestions. I've done some research about shrinkage and i got into two approaches: Ledoit-Wolf and Oracle Approximating Shrinkage Estimator. Do you know which one is more appropriated for a small dataset? Commented Mar 31, 2021 at 20:02
• @joann2555 what is small here? The dimension of correlation matrices or the number of observations or both? I used LW approach for small datasets with a few variables in them Commented Mar 31, 2021 at 20:08
• @whuber maybe “rarely” is not a good term but if you just generate a matrix with iid numbers in it is unlikely to be singular. That’s why I say you need to have something special about the way you generate the matrix Commented Mar 31, 2021 at 20:10
• Let's say small by a dataset with 24 features and 2000 samples (4 class, 500 samples per class). Commented Mar 31, 2021 at 20:46