Timeline for What is the final goal of RobustPCA methods?
Current License: CC BY-SA 4.0
23 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 6, 2018 at 5:29 | comment | added | SANN | Let us continue this discussion in chat. | |
| Aug 6, 2018 at 5:13 | comment | added | Has QUIT--Anony-Mousse | Central Limit Theorem. Yes, PCA is invertible, so from this point of view, no information is lost. But as you can see in the unit cube example, the original axes can be valuable, and this information is encoded in the projection; it cannot be recovered easily from the projected data. Same holds for MD: at least p points are completely encoded in the weight matrix, and the MDs do not provide differentiation of these points. | |
| Aug 6, 2018 at 3:42 | comment | added | SANN | And just as you know, PCA doesn't change the intrinsic manifold of the data and so the MD is not yielding an unreliable result. Even outlier detection methods using MD are working well for data containing clusters with convex shapes. | |
| Aug 6, 2018 at 3:32 | comment | added | SANN | @Anony-Mousse: Thank you for informing me about such phenomenon! I just created a uniform random matrix and applied PCA on it and checked the histograms! They just made me disappointed! but what do you mean with CLT?! I didn't find anything useful on google search. | |
| Aug 5, 2018 at 21:30 | comment | added | Has QUIT--Anony-Mousse | More formally, if you have uniform cube data with many dimensions, PCA is expected to produce components that approach normal distributions because of the CLT. So the PCA visualization may be misleading. | |
| Aug 5, 2018 at 21:23 | comment | added | Has QUIT--Anony-Mousse | That is an artifact by projecting with PCA. Since it mixes all dimensions, the resulting factors tend to appear more normal. So at that point, you already lost this information. | |
| Aug 5, 2018 at 20:26 | comment | added | SANN | @Anony-Mousse: You may not believe it! but almost all of those real data sets that I used in my research, were following Gaussian distribution in their clusters! You can test it on your own! just do a clustering on a real data set according to the prior knowledge available about its intrinsic distribution (like the number of classes in it) and after applying PCA on each cluster and plotting the histogram of each feature of the class separately, you will see the almost Normality in the distribution of the class! It's really awesome ;-) ... | |
| Aug 5, 2018 at 20:16 | comment | added | Has QUIT--Anony-Mousse | Plus, MD based on clusters - that is essentially EM outlier, where you use gaussian mixture model probabilities as outlier scores. Works well on Gaussian toy data, but much less on real data. Just to point you towards another prior work. | |
| Aug 5, 2018 at 20:14 | comment | added | Has QUIT--Anony-Mousse | Well, MD is sensitive to the quality of PCA, and PCA is sensitive to outliers, so the whole method may be problematic. Existing work may provide some ideas for these problems. | |
| Aug 5, 2018 at 18:19 | comment | added | SANN | @Anony-Mousse: Thank you for your advice on the issue, but the problem is that the whole method is based on the MD criterion and I cannot just disregard it for a little part of the proposed approach! | |
| Aug 5, 2018 at 17:59 | comment | added | Has QUIT--Anony-Mousse | You can use the k nearest neighbors of each point, with k = 3p then you'll always have enough at least for a rough estimate (you may want to look at the COP outlier method). But still, I would rather not use local Mahalanobis in this case. | |
| Aug 5, 2018 at 16:37 | comment | added | SANN | @Anony-Mousse: Yeah! you're right about MD that is not so viable for too little data and the traditional methods for outlier detection based on the MD may become degenerate. As I'm just disregarding those clusters in my research with n<=p and not using such methods like pseudo-inverse or anything like that! and waiting for them to grow as the data are coming. Because I'm afraid that the final results might become unreliable. But using pseudo-inverse of covariance matrix is still left as an option! ;-) | |
| Aug 5, 2018 at 7:38 | comment | added | Has QUIT--Anony-Mousse | Methods that are robust to outliers won't help if you already have too little data! Singularity is only the symptom (that can be trivially avoided by adding a tiny constant to the diagonal), the cause is an underspecified problem that allows for perfect overfitting. Mahalanobis cannot work with too little data. | |
| Aug 4, 2018 at 15:44 | comment | added | SANN | @user603: That was a useful comment! thanks a lot ... | |
| Aug 4, 2018 at 14:07 | comment | added | user603 |
@nilāmbara: robust statistics is a collection of methods designed to fit the majority of the data (as opposed to all of it or in the case of clustering arbitrarily sized chunks of it). The wiki article is a good place to start. Think of counterparts to classical parametric statistics toolbox (regression, MD, ...) that are not influenced much by outliers (one example is the relationship between trimmed means and classical means).
|
|
| Aug 4, 2018 at 14:03 | comment | added | user603 | The solution to your primary problem (computing the MD of singular data) is addressed in many places here (use the search button). See this one for example. Also, many clustering algorithms have deeper, more fundamental issues with singular data than just computing MD's (unbounded likelihood...). But this has nothing to do with robust statistics. Because of this, I would mention that robust statistics and clustering algos (or vise versa) solve intrinsically different issues (spoons and forks comes to mind). Hope this helps! | |
| Aug 4, 2018 at 13:57 | comment | added | mnm | @user603 what exactly is "robust statistics"? | |
| Aug 4, 2018 at 12:23 | comment | added | SANN | @user603: The question is that I have a scalable clustering problem that is using the MahalDist Criterion for assignment of each object to a cluster (you can find more inf. about my research here). The problem happens when I'm trying to assign a point to a cluster using MahalDist as long as its covariance matrix is singular and I just somehow heard that RobustPCA would be helpful! Thanks a lot ... | |
| Aug 4, 2018 at 11:01 | comment | added | user603 | I think I can answer b/c I work in that precise area, but I cannot make heads or tail from your question. Can you try to reformulate what your question is? What problem are you trying to solve? | |
| Aug 4, 2018 at 8:44 | comment | added | SANN | @Frans Rodenburg: Yeah! you're right about when the number of discovered clusters is larger than the actual number of underlying clusters. But there is another reference named "Robust principal component analysis?" that is claiming on the issue and makes me confused on the mentioned problem that is it useful for overcoming the singularity problem or not?! Thank you anyway ... | |
| Aug 4, 2018 at 8:27 | comment | added | Frans Rodenburg | Not experienced in this, so I'll just write a comment: PCA finds the largest variance among linear combinations of the input (PCs). However, since the variance is proportional to the sum of squared distances from the mean, an outlier squared may cause you to find PCs that only have the highest variance because of these outliers. So I agree with you that it should not solve the singularity problem, which can still occur e.g. if the specified number of clusters is larger than the actual number of underlying clusters. | |
| Aug 4, 2018 at 8:15 | review | First posts | |||
| Aug 4, 2018 at 9:16 | |||||
| Aug 4, 2018 at 8:14 | history | asked | SANN | CC BY-SA 4.0 |