0
$\begingroup$

I am building some experiments using the multivariate normal probability density function to estimate the likelihood of a given sample to come from a distribution. For that, the PDF is built using as parameter the covariance matrix of data.

In my experiment, i am presencing an error when calculating the PDF, it warns me that i'm trying to do a division by zero. Looking at the formula of the MVPDF, the only parameter that can be zero in the denominator is the determinant of the covariance matrix. My hypothesis is that the correlation matrix is singular and its determinant is 0.

I'd like to know if this has any relation with the size of my dataset, and (if so) how can i workaround this problem.

$\endgroup$

1 Answer 1

0
$\begingroup$

Yes, but...

The sample correlation matrix is usually extremely noisy. There's this whole area of statistics on "cleaning" correlation matrices, e.g. see RMT approach here or a shrinkage method.

However, random matrices rarely have determinant equal zero unless you construct them in certain ways. So, you might be having some other issue on top of the noise. I'd take a closer look and try to diagnose the exact cause. Otherwise, you can always apply the above shrinkage to shut the problem with zero determinant, as long as you know it's not the red herring

$\endgroup$
8
  • 1
    $\begingroup$ Re "rarely have determinant zero:" This might be an exaggeration, especially in some domains where such data occur all the time, such as chemical compositions (which, since they are known to sum to 100%, may be standardized to do so) as well as any other compositional data, such as exhaustive summaries of responses to multiple-choice questions. $\endgroup$
    – whuber
    Commented Mar 31, 2021 at 19:29
  • $\begingroup$ Thanks for the great suggestions. I've done some research about shrinkage and i got into two approaches: Ledoit-Wolf and Oracle Approximating Shrinkage Estimator. Do you know which one is more appropriated for a small dataset? $\endgroup$
    – joann2555
    Commented Mar 31, 2021 at 20:02
  • $\begingroup$ @joann2555 what is small here? The dimension of correlation matrices or the number of observations or both? I used LW approach for small datasets with a few variables in them $\endgroup$
    – Aksakal
    Commented Mar 31, 2021 at 20:08
  • $\begingroup$ @whuber maybe “rarely” is not a good term but if you just generate a matrix with iid numbers in it is unlikely to be singular. That’s why I say you need to have something special about the way you generate the matrix $\endgroup$
    – Aksakal
    Commented Mar 31, 2021 at 20:10
  • $\begingroup$ Let's say small by a dataset with 24 features and 2000 samples (4 class, 500 samples per class). $\endgroup$
    – joann2555
    Commented Mar 31, 2021 at 20:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.