It is a well known fact that (Making sense of principal component analysis, eigenvectors & eigenvalues) PCA can be understood as a transformation of axes given by the data when the data $X_1,\cdots,X_n$ are generated by $L^2$ random variables. Such an explanation collapse when the data generating mechanism does not have finite second moment. So in that case, what kind of drawbacks and potential morbidity does PCA have? And how can we understand PCA in that kind of situation?
$\begingroup$
$\endgroup$
2
-
2$\begingroup$ PCA is an analysis of real data, not of random variables having their theoretic distributions. Real data always have some mean and variance. $\endgroup$– ttnphnsSep 24, 2017 at 14:12
-
2$\begingroup$ @ttnphns I understand empirical moments are always finite, however consider dataset generated by Cauchy and the asymptopia of PCA, in such cases those 'data analysts' just running without test. $\endgroup$– Henry.LSep 24, 2017 at 14:46
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
Xie and Chen have a paper on Cauchy PCA (https://arxiv.org/pdf/1412.6506) which is explicitly intended for use with information that has undefined or infinite variance. One of the nice things about this paper is that they introduce the approach by discussing four patterns to data -- sparse small noise, sparse large noise, dense small noise and dense large noise -- and the assumptions appropriate for each: Gaussian PCA for small noise, Laplace PCA for sparse noise, probabilistic PCA for dense noise and Cauchy PCA as a robust solution for all noise patterns. This typology makes a lot of sense to me. In addition and upon request, the authors will provide some simple Matlab code for implementation.
answered Sep 24, 2017 at 14:46
-
3$\begingroup$ +1 obviously. Comment: you might want to first consider something less-exotic to start with. Robust PCA approaches essentially safe-guard against the presence of outliers that might render higher order moments irrelevant. There are very well-studied approaches on the matter for quite a while (eg. See Rousseeuw and van Driessen (1999) A fast algorithm for the minimum covariance determinant estimator) and implementations of them have been a staple of major R packages (eg.
MASS::cov.rob,robustbase::covMcd, etc.) $\endgroup$ Sep 24, 2017 at 15:06