5
$\begingroup$

In Andrew Ng's Machine Learning Coursera Class, he covers anomaly detection in multiple dimensions for both independent univariate Gaussians and multivariate Gaussians, the latter being more costly than the former.

Would running independent anomaly detection after orthogonalizing the data produce the same results as a multivariate anomaly detection? Is PCA too costly for this to ever be worthwhile (assuming it works at all)?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

No. PCA is performed on the multivariate case as well (there is an inverse of a covariance matrix in the formula), but it measures the distance from the mean using all dimensions. By looking at each axis individually (I am assuming thats what you mean when you say independent anamoly detection) you will miss anomalies that come about as a result of two or more dimensions acting awry.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ The point of the question is that a multivariate Gaussian with zero covariances (and whitening with PCA would ensure zero covariances) is equal to a product of univariate Gaussians. I am not sure how your answer addresses that. $\endgroup$
    – amoeba
    Dec 26, 2014 at 19:46
  • 1
    $\begingroup$ My answer was under a different premise that was stated within. If what you said is the case, then the two methods are equivalent and their is no cost difference. $\endgroup$
    – Malz
    Dec 26, 2014 at 19:57
  • $\begingroup$ If you read the OP again, you will see that it clearly asks if "independent anomaly detection after orthogonalizing the data [with PCA] produce[s] the same results as a multivariate anomaly detection [before orthogonalizing]". The second sentence of your answer is only true if you read this question leaving out the part I made bold. However, I think that your comment provides an adequate answer: "the two methods are equivalent and their is no cost difference". If you update your answer to reflect it, I will be happy to upvote! Cheers. $\endgroup$
    – amoeba
    Feb 25, 2015 at 13:18
-1
$\begingroup$

It should produce the same result, since all you are doing to rotate and move your reference system so as to center the data.

When doing it with PCA you would still need to compute a eigenvectors of covariance matrices, so it won't be cheaper.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I am not sure thus actually answers the original question, as it was about univariate vs. multivariate anomaly detection. $\endgroup$
    – amoeba
    Aug 26, 2014 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.