I am working with multivariate Gaussian distribution (MVG) to detect anomalies in time-series data. I have two features and thousands of observations. The first feature (left) has a very good normal distribution after applying "moving-averages" and Z-score normalization. However, the second feature has many zeroes and doesn't follow the normal distribution. Usually, my anomalies exist where feature-1 with a high score and feature-2 with a low score (especially zero). If it finds an outlier that has high scores in both features, it wouldn't call it. Because, in theory, MVG uses the covariance between feature1 and 2 and calls the predictions. Now my concerns are.

  1. Feature-2 zeroes are important to find outliers and I don't want to remove them. If I don't, it violates the normal distribution assumption. What should I do in this case?

  2. Also, in some rare cases where feature-1 and 2 have the highest scores, it detects outliers. Does it mean MVG is not working well or the first concern is somehow affecting how well the model works?

enter image description here

  • $\begingroup$ You pretty clearly don’t have multivariate Gaussian data. Why is the MVG assumption so crucial? $\endgroup$ – Dave May 29 '20 at 22:42
  • $\begingroup$ Because the model works well with data with a normal/gaussian distribution. $\endgroup$ – ferrelwill May 29 '20 at 23:20
  • $\begingroup$ You violate that assumption or normality. However, there are many misconceptions about when normality is required. What model are you using? $\endgroup$ – Dave May 29 '20 at 23:31
  • $\begingroup$ I am using EllipticEnvelope from sklearn link $\endgroup$ – ferrelwill May 29 '20 at 23:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.