I have calculated Hotelling's T2 statistic for detection of outliers in PCA analysis in Matlab. However, I am unsure as to whether or not it is a robust approach to remove these outliers? The output will be used in a cluster analysis and I am wondering if I remove the outliers, am I fundamentally changing the outcome of the cluster analysis in a very subjective way? ie. the outliers could be the thing that makes a specific data point belong to a specific group. So, my question is: Is it generally the rule that outliers are removed from PCA? I see a lot online about identifying them, but not so much about what to do with them afterwards.
$\begingroup$
$\endgroup$
0
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
As a very general rule, the proper treatment of outliers depend on the analysis purpose - if you're looking for large-scale tendencies, they often better be removed, but sometimes your goal might be actually finding the non-typical data points. Now, going into details... PCA is a method based on correlation/covariance matrix, which can be - at least in theory - very sensitive to outliers. One way to get around this problem is to use a robust correlation matrix - see e.g. https://cran.r-project.org/web/packages/robust/robust.pdf or, if you use Matlab, maybe https://sourceforge.net/projects/robustcorrtool/ will be better. Clustering can also serve as a outlier detection technique, but if you want to identify a few groups of similar points in the dataset, I'd suggest removing the outliers since - again - they can affect the workings of some clustering algorithms (like k-means, which is based on within-cluster variance) and make the results harder to interpret. We usually like the elements of a cluster to be rather homogeneous, and the very definition of outlier contradicts similarity to other cluster members... EDIT: sorry for not making myself clear - if you base PCA on robust covariance, there's no need to remove anything. When using 'ordinary' correlation, you might compare the results with and without outliers and see if they make any real difference.
-
1$\begingroup$ I like to say "classify outliers" instead of remove them. If you can't root cause them then how do you know they aren't data? I like the clustering approach, because PCA is really about the axes of a least-squares hyper-ellipsoid fit, and if there is no hyper-ellipsoid, then it isn't necessarily as useful. Clustering, especially using something like GMM (big bro of k-means, non-constant variance) helps improve the relationship between model and reality. $\endgroup$ Commented Jul 20, 2016 at 20:58
-
$\begingroup$ Good point on root cause identification :) $\endgroup$ Commented Jul 20, 2016 at 20:59