2
$\begingroup$

I have this data set of 12 input variables with a size of over 9000. Wanted to find any outliers using mahalanobis distance. As a central point, I decided to take mean and median of all observations, getting two central points. When I calculate the distance between the centre and datapoints using scipy, I get a uniform value of root 2 across all points. I even tried by implementing the distance formula in python, but the results are the same. I have also checked every step, including the inverse covariance, where I had to use numpy's pinv due to singular matrix . 1) Is it because of mean/median, what should be the central point ? 2) Is it saying something about the data set, most data are unique. 3) Is there something I am failing to understand. I am a stats amateur, so any insight is appreciated.

$\endgroup$
1
$\begingroup$

Barring programming errors, that would mean that $$(x_i - \mu)^T\Sigma^{-1}(x_i-\mu) = 2$$ for all points, which seems highly unlikely. It is also surprising that the inverse does not exist (unless some of those variables are exact linear combinations of the other), as you have more that 9000 observations and only 12 variables.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.