5
$\begingroup$

In Andrew Ng's Machine Learning Coursera Class, he covers anomaly detection in multiple dimensions for both independent univariate Gaussians and multivariate Gaussians, the latter being more costly than the former.

Would running independent anomaly detection after orthogonalizing the data produce the same results as a multivariate anomaly detection? Is PCA too costly for this to ever be worthwhile (assuming it works at all)?

$\endgroup$
2
$\begingroup$

No. PCA is performed on the multivariate case as well (there is an inverse of a covariance matrix in the formula), but it measures the distance from the mean using all dimensions. By looking at each axis individually (I am assuming thats what you mean when you say independent anamoly detection) you will miss anomalies that come about as a result of two or more dimensions acting awry.

$\endgroup$
  • 1
    $\begingroup$ The point of the question is that a multivariate Gaussian with zero covariances (and whitening with PCA would ensure zero covariances) is equal to a product of univariate Gaussians. I am not sure how your answer addresses that. $\endgroup$ – amoeba Dec 26 '14 at 19:46
  • 1
    $\begingroup$ My answer was under a different premise that was stated within. If what you said is the case, then the two methods are equivalent and their is no cost difference. $\endgroup$ – Malz Dec 26 '14 at 19:57
  • $\begingroup$ If you read the OP again, you will see that it clearly asks if "independent anomaly detection after orthogonalizing the data [with PCA] produce[s] the same results as a multivariate anomaly detection [before orthogonalizing]". The second sentence of your answer is only true if you read this question leaving out the part I made bold. However, I think that your comment provides an adequate answer: "the two methods are equivalent and their is no cost difference". If you update your answer to reflect it, I will be happy to upvote! Cheers. $\endgroup$ – amoeba Feb 25 '15 at 13:18
-1
$\begingroup$

It should produce the same result, since all you are doing to rotate and move your reference system so as to center the data.

When doing it with PCA you would still need to compute a eigenvectors of covariance matrices, so it won't be cheaper.

$\endgroup$
  • $\begingroup$ I am not sure thus actually answers the original question, as it was about univariate vs. multivariate anomaly detection. $\endgroup$ – amoeba Aug 26 '14 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.