# How to use non positive definite covariance matrix in multivariate Gaussian distribution

First of all sorry but I am a little bit of a newbie to advanced stats hence why my question may sound silly.

I am trying to perform a classification task where I assume that my data is generated by a multivariate Gaussian distribution. For that purpose I estimate the covariance matrices from my sample data, but for some of the variables I am getting a non positive definite matrix. Could anyone provide me with a sound explanation to this (I understand I have some non linearly independent variables in my data probably?) and maybe a workaround so I can proceed?

• Could you explain what you mean by "proceed"? What exactly is the matter with obtaining a singular covariance matrix?
– whuber
Oct 11, 2016 at 16:24
• Hi @whuber, in order to "proceed" with my classification task I need to estimate probabilities that depend on these estimated covariance matrices. What I mean is I am using a multivariate Gaussian to calculate the probabilities of an observation given each class and then just using the argmax of that to classify, hence why I need those covariance matrices to work for the probability density function... Any idea how I could work around my data to avoid singular covariance matrices? Oct 11, 2016 at 23:16
• It's unclear how the singularity of a covariance matrix could prevent you from estimating a probability.
– whuber
Oct 12, 2016 at 12:50
• Any data generated by a multivariate Gaussian function must have a positive definite covariance matrix. I am reversing that, and estimating the covariance matrices from my data set, having as a starting point the assumption that my data is Gaussian. If my covariance matrix is not positive definite, I cannot use it to calculate probabilities with a multivariate Gaussian. But maybe I am missing something in this approach...? Oct 12, 2016 at 14:06
• Any data generated by any distribution--including Gaussians--only need have a positive semidefinite covariance. You definitely can use your estimate to compute probabilities. After all, your covariance estimate (along with an estimated multivariate mean) completely determines the distribution!
– whuber
Oct 12, 2016 at 14:14