1
$\begingroup$

I have an ongoing research that is about clustering and for assigning each data point to a specific cluster, I'm using the Mahalanobis criterion that is gravely based on the covariance matrix of the cluster. As everyone knows, if the cardinality of the cluster is less than or equal to its dimensionality, then the covariance matrix will be singular and so, the Mahalanobis distance wont be reliable anymore!

But the thing is that I've heard that it would be possible to conquer such problems by using RobustPCA methods. As it's mentioned in this ROBPCA method, in the third paragraph of its Introduction section:

The goal of robust PCA methods is to obtain principal components that are not influenced much by outliers

I think it's trying to say that RobustPCA methods are only useful when we want to draw out the mere principle components that are obtained through real data points but not grossly corrupted observations. So, it's not OK to say that we can use it for overcoming the Mahalanobis criterion issue that is caused by singularity problem of the covariance matrix of the cluster!

However, is this correct?!

Regards ...

Improve this question
$\endgroup$
21
  • $\begingroup$ Not experienced in this, so I'll just write a comment: PCA finds the largest variance among linear combinations of the input (PCs). However, since the variance is proportional to the sum of squared distances from the mean, an outlier squared may cause you to find PCs that only have the highest variance because of these outliers. So I agree with you that it should not solve the singularity problem, which can still occur e.g. if the specified number of clusters is larger than the actual number of underlying clusters. $\endgroup$
    – Frans Rodenburg
    Commented Aug 4, 2018 at 8:27
  • $\begingroup$ @Frans Rodenburg: Yeah! you're right about when the number of discovered clusters is larger than the actual number of underlying clusters. But there is another reference named "Robust principal component analysis?" that is claiming on the issue and makes me confused on the mentioned problem that is it useful for overcoming the singularity problem or not?! Thank you anyway ... $\endgroup$
    – SANN
    Commented Aug 4, 2018 at 8:44
  • $\begingroup$ I think I can answer b/c I work in that precise area, but I cannot make heads or tail from your question. Can you try to reformulate what your question is? What problem are you trying to solve? $\endgroup$
    – user603
    Commented Aug 4, 2018 at 11:01
  • $\begingroup$ @user603: The question is that I have a scalable clustering problem that is using the MahalDist Criterion for assignment of each object to a cluster (you can find more inf. about my research here). The problem happens when I'm trying to assign a point to a cluster using MahalDist as long as its covariance matrix is singular and I just somehow heard that RobustPCA would be helpful! Thanks a lot ... $\endgroup$
    – SANN
    Commented Aug 4, 2018 at 12:23
  • 1
    $\begingroup$ The solution to your primary problem (computing the MD of singular data) is addressed in many places here (use the search button). See this one for example. Also, many clustering algorithms have deeper, more fundamental issues with singular data than just computing MD's (unbounded likelihood...). But this has nothing to do with robust statistics. Because of this, I would mention that robust statistics and clustering algos (or vise versa) solve intrinsically different issues (spoons and forks comes to mind). Hope this helps! $\endgroup$
    – user603
    Commented Aug 4, 2018 at 14:03

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.