I have a data set and I want to cleaning up my data set from the ouliers, so I decide to use the Mahalanobis distance to find the outliers. But I have a problem here since my covariance matrix isn't positive definite, and thats effected the result negative so I can't use the square root. So Can I still use mahalanobis method for this problem? If the answer is yes, how? What kind of alternative method to remove the outlier for this problem?
$\begingroup$
$\endgroup$
0
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
All covariance matrices are positive semi-definite. If the covariance matrix isn't positive definite, that implies that it is singular.
In this case, either or both of these is true:
A singular covariance matrix occurs when the source data contains linearly dependent features. You need to remove linearly dependent features from your source data. The gold-standard way to verify the rank of your feature matrix is to perform SVD. If the number of positive singular values is smaller than the number of features, it's rank-deficient.
Your matrix is numerically singular. This can happen even if all of the columns are linearly independent. You can verify this by checking the condition number of the matrix. You state that you have the case that $(x-y)^\top\Sigma^{-1}(x-y)<0$; if the results are only "slightly" negative, then this is a result of accumulated numerical error. (If the results are very negative, there's a large programming error somewhere.)
You could try modifying $\Sigma$ to enforce non-singularity. A common way to do this is to compute $\Sigma=USV^\top$ the singular value decomposition of $\Sigma$, and then "adjust" the singular values $S_{ii}$ so that they are pinned to some small, positive multiple of the largest singular value: $\tilde{S_{ii}}=\max\left\{S_{ii}, \epsilon \max{S_{ii}}\right\}$. A reasonable choice is $\epsilon=10^{-6}$ for double-floats. This will distort your covariance matrix a little bit, but alleviate your negative distance problem.